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I.         INTRODUCTION 

A.  BACKGROUND 

The  mission  of  United  States  Pacific  Command  (USPACOM)  is  to  enhance 

security  and  promote  peaceful  development  in  the  Asia-Pacific  region  by  deterring 

aggression,  responding  to  crisies  and  fighting  to  win  in  support  of  our  national  security 

objectives.  The  U.  S.  Commission  on  National  Security/2 1st  Century,  co-chaired  by 

Senators  Gary  Hart  and  Warren  Rudman  (2000),  recently  stated  that  among  other  issues: 

It  is  a  critical  national  interest  of  the  United  States  that  no  hostile  power  establish 
itself  on  U.S.  borders,  or  in  control  of  critical  land,  air,  and  sea  lines  of 
communication,  or—in  today's  new  world—in  control  of  access  to  outer  space  or 
cyberspace.  It  is  a  critical  national  interest  of  the  United  States  that  no  hostile 
hegemon  arise  in  any  of  the  globe's  major  regions,  nor  a  hostile  global  peer  rival  or  a 
hostile  coalition  comparable  to  a  peer  rival. . . 

Additionally,  the  commission  stated  that  the  security  of  allies  and  friends  is  a  critical 

national  interest  of  the  United  States  and  that  one  of  the  key  objectives  is  to  assist  in  the 

economic  and  political  integration  of  China,  India  and  Russia  into  the  mainstream  of  the 

international  community  (Hart  and  Rudman,  2000).   USPACOM' s  Theater  Engagement 

Plan  (TEP)  contributes  to  the  accomplishment  of  this  mission  by  planning,  coordinating, 

and  implementing  peacetime  military  activities  to  shape  the  region's  security 

environment. 

The  design  of  USPACOM's  TEP  is  to  identify  realistic  endstates  that  support  the 

prioritized  regional  objectives  for  the  Area  of  Responsibility  (AOR),  plan  those  activities 

that  support  accomplishment  of  the  approved  endstates  and  coordinate  the  components' 

efforts  to  shape  the  Asia-Pacific  regional  security  environment.  Engagement  planning 


helps  foster  the  development  of  security  communities  by  focusing  on  three  themes: 
goodwill,  access,  and  competent  coalition  partners. 

USPACOM's  engagement  strategy  for  the  AOR  is  to  enhance  basic  goodwill 
relations,  improve  access  for  U.S.  forces  and  develop  competent  coalition  partners  in 
order  to  facilitate  the  development  of  security  communities,  which  in-turn,  will  enhance 
security  and  promote  peaceful  development  in  the  Asia-Pacific  region. 
B.  STATEMENT  OF  THE  PROBLEM 

The  current  TEP's  guidance  and  assistance  policy  focuses  on  the  engagement  of 
Security  Communities  within  its  theater.  These  Security  Communities  are  defined  as 
groups  of  nations  that  don't  plan  or  want  to  fight  each  other,  that  cooperate  in  the 
peaceful  resolution  of  disputes,  and  that  work  together  on  other  common  issues.  These 
groups  of  nations  are  considered  neither  a  defense  community  nor  a  security  alliance. 

As  previously  stated,  USPACOM's  engagement  strategy,  as  it  concerns  these 
security  communities,  focuses  on  three  entities:  goodwill  relations,  access  (training  and 
contingencies),  and  regional  readiness.  Regional  readiness  is  the  mechanism  for 
developing  competent  coalition  partners  and  is  mainly  concerned  with  interoperability 
issues  such  as  tactics,  techniques  and  procedures  and  foreign  military  sales.  Regional 
readiness  is  also  concerned  with  combined  operations  involving  humanitarian  issues. 

The  engagement  strategy  of  each  individual  country  focuses  on  four  areas  of 
engagement.  USPACOM  is  interested  in  utilizing  a  coordinated  focus  approach  in  order 
to  expand  current  engagement  activities.  USPACOM  is  also  interested  in  continuing 
existing  relationships  to  foster  current  and  future  engagement  activities.  Additionally, 


USPACOM  is  interested  in  increasing  the  potential  efficiencies  in  order  to  tailor  current 
and  future  engagement  activities  to  achieve  specific  objectives.  Finally,  USPACOM  is 
interested  in  developing  a  way  to  effectively  interact  with  countries  that  possess 
engagement  restrictions.  These  engagement  restrictions  may  exist  due  to  various 
political,  military  and/or  economic  considerations. 
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Figure  1.1-  USPACOM  Engagement  Planning  Cycle. 
The  endstates  and  objectives  of  the  theater  engagement  activities  are  put  forth  by 
the  USPACOM  Commander-in-Chief  (CINC)  and  are  not  consider  negotiable.  However, 
the  ends  and  means  of  achieving  these  objectives  are  somewhat  negotiable  between  the 


joint  staff  members  and  USPACOM's  subordinate  and  supporting  agencies. 

The  USPACOM  engagement  planning  cycle  is  a  12-month  cycle  that  involves 
inputs  from  numerous  and  various  internal  and  external  sources.  As  shown  in  Figure  1.1, 
the  Engagement  Working  Groups  (EWGs),  consisting  of  country  representatives,  meets 
four  times  throughout  the  year  and  plan  projected  engagement  activities  for  forthcoming 
fiscal  years.  This  planning  cycle  consists  of  the  coordination,  interaction,  and  decision 
making  of  35  separate  internal  and  external  staff  and  support  entities  in  the  pursuit  of  the 
USPACOM's  engagement  goals  and  objectives  and  is  projected  out  for  seven  fiscal 
years. 

The  engagement  activities  employed  by  USPACOM  are  categorized  by  type  and 

consist  of  12  separate  areas,  which  fall  under  eight  major  TEP  categories.  These  major 

TEP  categories  are  as  follows: 

Operational  Activities 
Exercises 

Combined  Training 
Combined  Education 
Military  Contacts 
Security  Assistance 
Humanitarian  Assistance 
Other  Engagement  Operations 

Under  these  main  categories,  USPACOM  is  interested  in  pursuing  12  types  of 

engagement  activities:  Counterdrug  and  Repatriation  operations  (Operational  Activities); 

Exercises;  Training;  Education;  High  Level  Visits,  Port  Visits,  Conferences,  and 

Exchanges  (Military  Contacts);  Education  and  Foreign  Military  Sales  (Security 

Assistance);  Humanitarian  Assistance;  and  Other  Engagement  Operations.  It  should  be 


noted  that  the  quantity,  level  and  type  of  engagement  activity  chosen  is  dependent  on  the 
intricacies  of  the  particular  country  being  considered. 

USPACOM  J56  (Future  Plans  and  Operations,  Engagement)  has  a  requirement  to 
assess  the  effectiveness  of  engagement  activities  proposed,  planned  and  conducted  within 
USPACOM's  AOR.  Although  the  CINC's  goals  and  objectives  for  the  engagement 
process  have  been  formally  defined,  there  exists  no  formal  process  to  link  the 
engagement  activities  and  their  effectiveness  to  the  goals  and  objectives.  Consequently, 
there  currently  is  no  way  to  gauge  the  impact  of  the  most  significant  engagement  activity 
or  comparatively  assess  the  value  of  one  activity  type  versus  another  in  the  achievement 
of  a  particular  goal  or  objective. 
C.  SCOPE 

The  aim  of  this  work  is  to  develop  a  basic  understanding  of  what  decision  analysis 
and  decision  support  aides  can  do  for  decision-makers,  their  usefulness,  appropriateness, 
and  some  of  their  limitations  in  the  context  of  engagement  activity  selection  and 
evaluation. 

This  thesis  will  begin  with  a  limited  review  of  decision  analysis,  decision  models, 
and  decision  support  systems  in  order  to  provide  a  basic  foundation  for  other  concepts 
presented  in  later  sections.  This  work  will  also  briefly  look  at  the  function  and  utility  of 
different  model  types  that  may  be  useful  to  decision-makers  in  the  process  of  decision 
analysis  and  alternative  selection.    Additionally,  this  thesis  will  provide  a  limited  review 
and  comparison  of  the  theoretical  basis  of  Multi-Attribute  Utility  Theory  (MAUT)  and 
the  Analytical  Hierarchy  Process  (AHP)  decision  analysis  and  support  methodologies  and 


how  they  may  be  applied  to  USPACOM's  engagement  activity  selection  problem. 

The  primary  focus  of  this  work  will  provide  a  limited  analysis  of  the  capabilities 
of  Expert  Choice™,  a  Commercial-Off-The-Shelf  (COTS)  software  application,  which 
employs  the  AHP  methodology.  The  purpose  of  this  analysis  will  be  to  determine  the 
usefulness  of  this  application  in  aiding  USPACOM's  decision-makers  in  selecting  the 
most  effective  theater  engagement  activities  from  a  list  of  available  alternatives. 


II.       REVIEW  OF  DECISION  THEORY,  MODELS,  AND  SUPPORT 

SYSTEMS 


A.  DECISION  THEORY 

A  good  decision  usually  results  from  the  application  of  logic,  review  of  all 
available  data  and  alternatives,  and  pursuit  of  a  quantitative  approach.  There  are  times 
when  a  decision,  initially  thought  to  be  good,  obtains  unfavorable  or  unexpected  results. 
Although  the  results  of  the  decision  were  unfavorable,  the  decision  is  still  a  good  one  due 
to  the  fact  that  the  decision  was  made  properly.  Conversely,  poor  decisions  that  fail  to 
apply  logic,  fail  to  consider  all  of  the  available  information,  fail  to  consider  all  of  the 
alternatives,  and  fail  to  apply  some  quantitative  approach  can  still  result  in  a  favorable 
outcome.  However,  a  decision  that  produces  favorable  results  through  the  use  of  faulty 
logic  and  incomplete  data  and  information  is  still  considered  a  poor  decision.  The 
application  of  some  of  the  basic  tenets  of  decision  theory  along  with  the  use  of  some 
decision  support  systems  will  increase  the  occurrence  in  which  the  decision-maker 
achieves  a  successful  outcome.  (Render  and  Stair,  1 997) 

Regardless  of  the  type  of  decision  to  be  made,  Render  and  Stair  (1997)  believe 
that  decision-making  is  a  six-step  process.  These  six  steps  are  as  follows: 

1 .  The  problem  is  clearly  defined. 

2.  All  possible  alternatives  are  listed. 

3.  Possible  outcomes  are  identified. 

4.  Each  combination  of  alternative  and  outcome  is  listed  with  respect  to  its 
expected  payoff. 

5.  Select  one  of  the  available  decision  support  theory  models. 

6.  Apply  the  model,  analyze  the  results  and  make  your  decision. 


The  amount  and  type  of  information  or  knowledge  they  have  about  the  particular 
situation  usually  influences  the  decision-making  of  most  leaders.    In  1960,  Dr.  Herbert 
A.  Simon  provided  an  early  descriptive  model  of  the  decision-making  process  that 
involved  the  use  of  three  major  sub-processes.  These  three  sub-processes  are  just  as 
sound  today  and  still  very  applicable.  The  sub-processes  are  as  follows: 

•  Intelligence  -  The  stage  where  the  decision-makers  scan  their  environment  for 
potential  problems/opportunities  that  will  require  the  making  and  execution  of 
decisions. 

•  Design  -  The  stage  that  consists  of  the  generation,  development,  analysis  and 
feasibility  assessment  of  potential  courses  of  action  or  alternatives. 

•  Choice  -  The  stage  where  one  course  of  action  or  alternative  is  selected  and 
eventually  implemented.  (Young,  1989) 

As  proof  of  their  validity  today,  compare  them  with  Young's  (1989)  statement  that 

Decision  theory  envisions  any  decision  problem  as  consisting  of  the  elements  of: 

•  Alternative  strategies  defined  as  courses  of  action  or  a  particular  combination 
of  "settings"  of  the  variables  under  the  control  of  the  decision-maker. 

•  A  combination  of  conditions  (particular  "settings")  of  relevant  variables 
which  are  not  controllable  by  the  decision-maker  (traditionally  called  "states 
of  nature"  although  the  conditions  could  be  man-made  by  persons  other  than 
the  decision-maker). 

•  Interactions  between  strategy  and  each  state  of  nature  that  result  in  outcomes 
of  importance  to  the  decision-maker  and  which  can  be  measured  in  some  form 
of  payoff  units  (often  money). 

•  A  criterion  or  analysis  rule  by  which  a  decision-maker  can  assess  the  situation 
and  select  a  particular  strategy. 

Regardless  of  the  composition  of  the  decision  problem,  this  decision-making 

usually  occurs  in  one  of  three  types  of  environments:  decision-making  under  certainty; 

decision-making  under  risk;  and  decision-making  under  uncertainty.  Decision-making 

under  certainty  involves  the  occurrence  where  decision-makers  know  the  consequences 

of  every  alternative  and  chooses  the  alternative  with  the  most  favorable  outcome. 


Decision-making  under  risk  concerns  decision-makers'  knowledge  of  the  probability  of 
each  alternative  and  their  desire  to  maximize  the  expected  result.  Lastly,  decision- 
making under  uncertainty  concerns  the  situation  where  decision-makers  have  no 
knowledge  of  the  probabilities  of  the  possible  outcomes.  (Render  and  Stair,  1997) 

Sage  (1991)  believes  that  a  fourth  environment,  decision  under  conflict,  exists  and 
that  it  is  more  appropriately  handled  with  game  theory  and  conflict  analysis.  Problems 
that  arise  in  the  environment  of  decision-making  under  the  conditions  of  certainty  are 
usually  addressed  by  deterministic  decision  theory.  Decision  analysis  models  are  usually 
appropriate  for  dealing  with  problems  associated  with  decision-making  under  risk  and 
decision  making  under  uncertainty.  The  selection  of  the  best  engagement  activity  from  a 
set  of  alternatives  is  a  problem  associated  with  risk  and/or  uncertainty. 
B.  DECISION  MODELS 

This  section  describes  the  system  engineering  approach  to  problem  solution  and 
the  modeling  techniques  proposed  by  Sage  (1991).  The  basic  guidelines  of  this  approach 
to  problem  solution  include: 

1 .  Definition  of  the  problem  or  issue  formulation. 

2.  Analysis  of  the  problem  or  issue. 

3.  Interpretation  of  the  analysis,  which  includes  alternative  evaluation  and 
selection  and  implementation. 

Under  the  first  guideline,  the  focus  is  on  the  definition  of  the  problem.  This 

formulation  process  includes  identification  of  the  individual  problem  elements  or 

characteristics.  Problem  definition  is  normally  a  group  activity  involving  those 

individuals  who  are  most  familiar  with  the  issue  at  hand  (e.g.,  engagement  working 

groups).  If  done  properly,  this  process  identifies  the  needs,  constraints,  alterables, 


political  factors,  and  military  considerations  affecting  a  particular  problem.  This  process 
should  also  serve  to  identify  the  relationships  among  these  elements. 

An  important  concern  in  this  process  is  the  identification  and  structuring  of  the 
goals  and  objectives  for  the  alternative  that  is  eventually  selected  by  the  decision-makers. 
Using  the  concept  of  collective  inquiry,  there  are  two  groups  of  methods  for  achieving 
this  goal.  The  first  group  includes  brainstorming,  synectics  and  nominal  group  technique, 
all  require  that  a  facilitator-led  group  meet  together  in  same  place  and  at  the  same  time. 
The  nominal  group  techniques  uses  the  step  process  of  idea  generation,  discussion,  and 
prioritization  and  is  generally  more  effective  than  brainstorming  in  reducing  the  influence 
of  dominate  personalities.  Synectics  is  based  on  problem  analogies  and  is  considered 
more  appropriate  for  the  generation  of  truly  innovative,  unconventional  ideas.  Also,  it 
normally  requires  a  more  experienced  facilitator  and  group.  Synectics,  along  with 
brainstorming  is  considered  to  be  directly  interactive  where  nominal  is  indirectly 
interactive.  Group  members  using  the  nominal  group  technique  may  not  communicate 
directly. 

The  second  group  of  collective  inquiry  includes  questionnaires,  survey  and 
Delphi.  Although  these  methods  do  not  require  the  gathering  of  the  group  members  at  a 
particular  time  and  place,  they  do  tend  to  take  more  time  to  complete.  With  no 
interaction  among  the  participants,  questionnaires  and  surveys  solicit  individual  answers 
to  questions  from  a  large  group  and  then  derive  the  overall  results.  Delphi  technique 
normally  requires  a  written  anonymous  response  over  several  rounds.  The  results  of  each 
previous  round  provide  feedback  to  the  participants  who  are  asked  to  comment,  revise 
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and/or  reiterate  their  views.  Although  this  technique  may  prove  to  be  highly  instructive, 
it  can  be  a  lengthy  process. 

Interaction  matrices,  trees,  structured  modeling,  and  casual  loop  diagrams  are 
some  of  the  many  available  element-structuring  aids  that  can  be  useful  during  this  step. 
Interaction  matrices  may  be  helpful  in  identifying  clusters  of  closely  related  elements  or 
identifying  the  couplings  of  elements  from  different  sets.  Trees  are  graphical  aids  that 
are  useful  in  the  portrayal  of  hierarchical  structures.  Structured  modeling  techniques  are 
usually  computer  aides  designed  to  assists  individuals  or  groups  in  structuring  large  sets 
of  elements.  Casual  loop  diagrams  provide  a  graphical  heuristic  of  the  casual  interactions 
between  sets  of  variables.  The  use  of  any  of  the  available  structuring  methods  can  lead  to 
greater  clarity  of  the  problem  as  well  as  the  identification  of  new  or  revised  descriptions 
of  the  problem  elements. 

The  second  guideline  proposed  by  Sage  focuses  on  problem  analysis  and 
generally  consists  of  two  steps.  The  first  step,  impact  assessment,  is  the  analysis  of  the 
alternatives  to  determine  their  impact  on  the  overall  goal  or  objective.  The  second  step 
involves  an  optimization  process,  which  attempts  to  maximize  the  alternatives' 
performance  in  terms  of  goal  satisfaction  within  the  stated  problem  constraints. 

Simulation  and  modeling  techniques  are  very  useful  for  completing  this  second 
step.  A  properly  constructed  model  has  the  ability  to  mimic  the  behavior  of  real  system 
in  a  way  that  allows  for  experimentation  with  alternatives  that  may  not  be  possible  with 
the  real  system.  Since  models  are  dependent  on  the  value  system  and  purpose  of  use,  it  is 
important  that  one  be  able  to  determine  the  correctness  of  the  model's  predictions  in 
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order  to  validate  it.  The  modeling  of  systems  enhances  our  ability  to  comprehend  and 
understand  the  fine  differences  in  the  interrelationships  of  the  system  elements  and  our 
own  relationship  to  them.  Modeling  gives  decision-makers  the  ability  and  opportunity  to 
look  at  a  problem  or  decision  from  several  different  viewpoints  and  perspectives.  The 
last  step  in  this  process  begins  with  the  evaluation  and  comparison  of  alternatives.  After 
one  of  the  alternatives  is  selected,  an  implementation  plan  is  then  designed.  Sage  points 
out  that  there  is  a  great  distinction  between  the  optimization  step  of  the  analysis  process 
and  the  evaluation  of  these  refined  alternatives.  Although  there  must  exist  more  than  one 
alternative  for  evaluation,  it  is  important  for  decision-makers  to  attempt  avoid  a  large 
number  of  biases  in  their  evaluation  and  decision-making.  It  is  the  interpretation  process 
of  evaluation  and  decision-making  that  involves  the  most  interactions  with  the  previous 
two  steps. 

Although  there  are  numerous  methods  available  to  facilitate  evaluation  and 
decision-making,  following  sections  and  chapters  will  focus  on  the  Multi-Attribute 
Utility  Theory  and  Analytic  Hierarchy  Process  methodologies.  MAUT  has  been 
designed  to  provide  a  methodology  for  comparing  and  ranking  alternatives  that  consist  of 
numerous  attributes  or  characteristics.  The  important  attributes  are  identified,  ordered 
and  assigned  a  relative  weight  or  utility  by  the  decision-maker.  The  measurement 
attributes  are  used  in  the  calculation  of  an  overall  utility  for  each  alternative. 
Additionally,  MAUT  allows  for  the  use  of  a  variety  of  utility  structures  and  gives 
decision-makers  the  ability  to  incorporate  their  attitude  toward  risk  in  the  utility  formula. 
(Sage,  1991) 
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The  AHP  application  provides  a  method  for  converting  subjective  judgments  into 
relative  values  in  the  absence  of  physical  or  statistical  measures.  AHP  decomposes 
problems  by  identifying  relative  factors,  making  comparative  judgments  on  these 
elements,  and  using  pairwise  comparison  matrices  to  determine  the  relative  importance 
and  overall  rating  of  the  available  alternatives.  (Olson,  1996) 

Young  (1989),  believes  that  decision-makers  can  use  the  benefits  of  modeling 
within  a  Decision  Support  System  (DSS)  to  help  them  deal  with  semi-structured  decision 
problems  more  effectively  and  efficiently.  Young  provides  the  following  list  of  potential 
benefits: 

1 .  To  more  fully  understand  the  implication  of  one's  own  judgments,  and  to 
modify  judgments  where  they  appear  to  inconsistent  with  one  another  or  with 
what  is  known; 

2.  To  aid  effective  advocacy  by  means  of  more  thorough  analysis  and  testing  of 
alternative  assumptions  and  strategic  alternatives; 

3.  To  identify  decision  problems  variables  to  which  critical  outcomes  appear  to 
be  most  sensitive,  so  that  further  efforts  at  information  gathering  and  analysis 
can  be  more  effectively  directed; 

4.  To  increase  the  speed  and  efficiency  of  analysis  so  that  more  alternatives  can 
be  examined,  thereby  increasing  the  likelihood  of  identifying  a  better  strategy; 

5.  To  more  efficiently  and  consistently  generate,  integrate,  and  judgmentally 
modify  forecasts  of  outcomes  needed  for  planning. 

Although  models  may  not  determine  the  single  best  strategy  for  a  particular  problem, 

they  do  provide  a  heuristic  algorithm  that  almost  always  provides  a  better  outcome  than 

routine  trial  and  error. 

C.  DECISION  SUPPORT  SYSTEMS 

The  evolution  of  DSS  over  the  past  twenty  years  has  firmly  established  the 

purpose  of  DSS  as  the  provider  of  resources  to  be  used  by  decision-makers  in  the 

decision-making  process.  The  DSS  are  available  in  many  makes  and  models  with 
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varying  capabilities.  However,  all  provide  information.  This  information  lends 

credibility  to  the  final  decision,  but  does  not  become  the  final  decision.  The  final 

decisions  that  include  data  from  the  DSS  and  many  other  factors  are  determined  by  the 

decision-makers. 

Sprague  and  Carlson  (1982)  have  defined  DSS  as: 

A  class  of  information  systems  that  draw  on  transaction  processing  systems  and 
interacts  with  other  parts  of  the  overall  information  system  to  support  the  decision 
making  activities  of  managers  and  other  knowledge  workers  in  organizations. 

Andriole  (1989)  agrees  with  this  statement.  As  Shown  in  Figure  2.1,  he  believes  that 

realm  of  decision  support  has  numerous  dimensions.  Actual  decisions  are  made  at  the 

center,  while  information  and  support  activities  that  contribute  to  the  evaluation  of 

options  and  selection  of  the  final  decision  reside  outside  in  concentric  circles.  There  are 

other  definitions  that  are  more  restrictive  while  still  others  appear  to  be  somewhat 

broader.  In  either  case,  Andriole  believes  that  there  are  many  similarities  between 

industry,  government  and  military  decision  support  systems. 

Decision  support  includes  all  of  the  supporting  data,  information,  activities  and 

expertise  required  to  arrive  at  the  final  selection  of  alternatives.  However,  decision 

support  is  not  completely  focused  on  alternative  selection.  Valuable  decision  support 

information  may  be  provided  by  other  decisions  that  have  no  direct  connection  with  the 

selection  of  alternatives.  (Andriole,  1989) 
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Figure  2.1  -  The  Range  of  Decision  Support  after  (Andriole,  1989). 
Hogue  and  Watson  (1983)  provided  that  the  criteria  for  a  DSS  should  include  the 
following  principles: 


Supports  but  does  not  replace  decision-making. 

Directed  toward  unstructured  and/or  semi-structured  decision-making  tasks. 

Data  and  models  are  organized  around  the  decisions. 

Easy  to  use  software  interface. 

Interactive  processing. 

DSS  use  and  control  is  determined  by  the  user. 

Flexible  and  adaptable  to  changes  in  the  environment. 

Quick  ad  hoc  DSS  building  capabilities. 
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A  DSS  has  the  capability  to  assist  decision-makers  with  many  types  of  decisions 

depending  on  system's  particular  emphasis.  DSS  have  been  used  to  support  general  long 

range  planning,  strategic  assessment,  operational  planning  and  control,  reports  and 

analysis,  and  general  budgeting  among  others  (Meador,  1984).  As  an  example  of  this, 

USPACOM  is  currently  developing  a  Theater  Engagement  Planning  Management 

Information  System  (TEPMIS)  to  assist  them  in  tracking  planned  and  executed 

engagement  activities  conducted  in  their  AOR.  Although  the  list  of  potential  applications 

continues  to  grow  unabated,  the  fundamental  benefits  of  using  a  DSS  have  remained 

relatively  constant.    The  main  benefits  of  DSS,  as  derived  from  numerous  studies,  are 

provided  below  in  descending  order  of  perceived  usefulness. 

Provide  information  processing  and  retrieval  capabilities. 

Evaluate  alternatives. 

Assist  in  identifying  problems. 

Assist  in  interpreting  information. 

Provide  real  time  analysis  of  current  problems  and  opportunities. 

Suggest  decision  alternatives. 

Provide  ability  to  ask  what  if  questions. 

Manage  executive  time  scheduling  daily  activities. 

Increase  decision  confidence. 

While  this  list  is  not  all-inclusive,  it  is  indicative  of  the  capabilities  desired  by  senior 

decision-makers  when  evaluating  or  justifying  the  use  of  such  a  system.  (Hogue,  et  al., 

1985;  Keen,  1981;  Meador,  et  al.,  1984;  Money,  et  al.,  1988) 

The  next  chapter  will  provide  a  basic  introduction  to  the  theoretical  foundations 

and  applications  of  MAUT  and  AHP.  It  is  not  the  author's  purpose  to  provide  a  complete 
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analysis  of  these  two  methodologies  and  all  of  their  potential  applications.  The 
fundamental  concepts  of  each  methodology  are  provided  in  order  to  provide  a  better 
understanding  of  the  capabilities  of  the  software  applications. 
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III.     ALTERNATIVE  SELECTION 

As  the  problem  concerns  the  selection  of  the  best  engagement  activity  to  achieve 
a  stated  goal  or  objective,  this  section  will  focus  on  two  methodologies  for  completing 
this  task.  Both  of  the  multiple  criteria  decision-making  aids  addressed  here,  MAUT  and 
AHP,  are  concerned  with  selecting  the  best  choice  from  a  given  set  of  alternatives. 
Although  MAUT  is  probably  the  more  theoretically  accepted  approach,  AHP  is  one  of 
the  more  popular  systems  in  use  (Olson,  1 996).  Before  looking  at  these  two 
methodologies,  we  must  address  a  problem  both  of  these  techniques  share,  one  of 
determining  the  numerical  value  of  a  subjective  judgment. 

A.  QUANTIFYING  SUBJECTIVE  JUDGMENTS 

Nearly  50  years  ago,  Stevens  (1951)  define  measurement  as  "the  assignment  of 
numbers  to  observations  according  to  a  set  of  rules."  The  assignment  of  numerical  values 
when  measuring  objects  infers  the  existence  of  a  scale  of  measurement.    There  are  four 
widely  accepted  and  commonly  used  measurement  scales:  nominal,  ordinal,  interval,  and 
ratio.  A  nominal  scale  is  simply  categorizing  a  set  of  data  into  mutually  exclusive 
subclasses.  The  data  contained  in  these  subclasses  are  counted  according  to  frequency  of 
occurrence.  There  is  no  implied  order  or  value.  In  an  ordinal  scale,  the  numbers  are 
assigned  to  the  various  instances  of  the  property.  This  scale  of  measurement  provides 
data  about  the  order  of  the  categories  but  does  not  indicate  the  level  of  the  differences 
between  them.  Interval  scales  have  all  the  properties  of  an  ordinal  scale,  and  in  addition 
the  distances  between  any  two  numbers  on  the  scale  are  equally  spaced.  This  scale  of 
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measurement  provides  an  identity,  an  order,  and  a  constant  unit  of  measure,  which  serves 
to  indicate  the  exact  value  of  an  instance  in  a  particular  category.  A  ratio  scale  has  all  the 
characteristics  of  an  interval  scale,  but  it  also  has  a  true  zero  point  as  its  origin  and  has 
the  property  of  proportionality.  (Rea  and  Parker,  1997) 

These  levels  of  measurement  are  used  in  the  process  of  manipulating  the  numbers 
into  a  meaningful  value  of  the  object  being  measured.  Steven  M.  Anderson  (1987),  a 
student  at  the  Naval  Postgraduate  School,  proposed  that  there  are  four  main  traditional 
methods  of  quantifying  subjective  judgments.  These  methods,  as  presented  in  his  thesis 
concerning  a  goal-programming  model,  are  described  on  the  following  pages  to  show 
their  potential  for  application  to  the  problem  at  hand. 

1.  NUMERICAL  RATING  METHOD 

The  first  method  presented  by  Anderson  is  the  numerical  rating  method,  a  very 
simple  and  direct  method  for  quantifying  subjective  judgments.  This  method  was  first 
proposed  by  Stevens  (1951),  as  a  method  of  obtaining  comparative  rankings  in 
psychophysical  experiments.  Using  this  method,  evaluators  are  asked  to  associate  rated 
items  with  fixed  reference  points  by  assigning  them  numbers,  or  by  plotting  them  as 
points  on  a  continuous  number  line.    An  example  of  this  technique  would  be  the 
consideration  of  two  engagement  activities  and  their  potential  effectiveness.  A  decision- 
maker w6uld  indicate  the  position  of  these  two  alternatives  with  respect  to  two  other 
alternatives,  one  that  was  not  very  effective  and  another  highly  effective  activity.  After 
recording  all  of  the  subjective  responses,  the  geometric  mean  of  the  numerical  estimates 
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is  computed.  The  continuous  number  line  representation  of  this  example  is  shown  in 
Figure  3.1. 


x X X X 

less  effective  activity  activity  1  activity  2     highly  effective  activity 


Figure  3.1  -  Numerical  Rating  Continuous  Line. 

The  primary  advantage  of  the  numeric  rating  method  is  the  simplicity  of  its 
computation.    It  is  easy  to  analyze  the  results  with  basic  statistics  and  test  for  significant 
differences.  The  application  of  a  scale  relationship  between  these  four  activities  results  in 
interval  scale  data  that  can  be  transformed  into  any  other  kind  of  scale.  The  main 
disadvantage  is  that  there  is  no  natural  origin  for  judgments,  and  the  evaluators  frequently 
disagree  with  the  positioning  of  reference  points.  There  are  also  no  bounds  on  the 
interval  scale;  the  lower  bound  (not  effective  situation)  of  the  engagement  activity 
example  is  set  to  zero,  but  there  appears  to  be  no  upper  bound. 

2.  CATEGORICAL  JUDGMENT  METHOD 

The  categorical  judgment  method,  the  second  method  proposed  by  Anderson,  is  a 
commonly  used  means  of  obtaining  numerical  ratings  from  subjective  ratings;  wherein 
evaluators  assign  instances  to  previously  ordered  categories.  For  example,  staff  members 
could  be  asked  to  rate  the  risk  of  a  particular  engagement  activity  according  to  a  scale  of 
high,  above  average,  average,  below  average,  and  very  low  as  shown  in  Table  3.1.  This 
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example  uses  25  staff  members.  The  number  of  categories  used  can  range  from  two  to 
nine  according  to  desires  and  resources  of  the  surveyor  and  the  skills  of  the  evaluators 
(Fink  and  Kosecoff,  1985). 


HIGH 

ABOVE 
AVG 

AVG 

BELOW 
AVG 

VERY 

LOW 

Activity  A 

9 

4 

6 

4 

2 

Activity  B 

7 

9 

7 

2 

0 

Activity  C 

5 

11 

8 

0 

1 

Table  3.1  -  Initial  Rankings  of  Constant  Sum  Method. 

The  item  in  question  is  rated  and  the  results  are  used  to  construct  a  matrix  of  the 
cumulative  frequency  data  with  n-row  instances  and  m-column  categories.  Each  entry  of 
this  matrix  represents  the  number  of  evaluators  who  rated  instance  i  in  category/  In  the 
cumulative  relative  frequency  matrix  Cy,  entries  are  the  proportions  of  the  evaluators 
rating  instance  i  in  or  below  category  j.  This  matrix  is  shown  in  Table  3.2. 


Cij 

HIGH 

ABOVE 
AVG 

AVG 

BELOW 
AVG 

VERY 
LOW 

Activity  A 

0.36 

0.16 

0.24 

0.16 

0.08 

Activity  B 

0.28 

0.36 

0.28 

0.08 

0.00 

Activity  C 

0.20 

0.44 

0.32 

0.00 

0.04 

Table  3.2  -  Constant  Sum  Method  Cumulative  Frequency  Matrix. 
The  elements  of  this  matrix  are  considered  as  areas  under  a  standard  normal  curve  and 
are  converted  to  the  corresponding  z-values.  These  z-values  are  then  recorded  in  a  zv- 
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matrix  (Table  3.3)  consisting  of  w-rows  and  (m-7)-columns.  The  rightmost  column  may 
be  omitted  for  computational  purposes. 


Zij 

HIGH 

ABOVE 
AVG 

AVG 

BELOW 
AVG 

Activity  A 

0.6406 

0.5636 

0.5948 

0.5636 

Activity  B 

0.6103 

0.6406 

0.6103 

0.5319 

Activity  C 

0.5793 

0.6700 

0.6255 

0.5000 

Table  3.3  -  Constant  Sum  Method  Standard  Normal  Matrix. 
The  row  average,  r,-,  and  column  average,  Cj,  are  calculated,  and  a  grand  average  G  is 

found  by  averaging  the  column  averages.  A  column  sum  of  squares  (SSC)  is  computed 
as  shown  in  Equation  3-1: 

SSC  =  ^(Cj-G)2  (3-1) 

j 

For  each  row,  the  row  sum  of  squares  (SSRj)  is  computed  using  Equation  3-2: 

SSR^^-rf  (3-2) 


The  scale  value  of  the  instances  s„  are  found  by  solving  Equation  3-3  in  each  row: 


SSC, 

V  ssr. 


(3-3) 


Again,  the  main  advantage  of  the  categorical  judgment  method  is  that  it  obtains 
values  with  interval  scale  properties  that  can  be  linearly  transformed  to  any  other  scale. 
This  method  promotes  the  use  of  straightforward  and  uncomplicated  questionnaires. 
Although  it  is  more  sophisticated  than  the  numerical  rating  method,  it  is  still  relatively 
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easy  to  compute.  The  major  disadvantage  of  this  method  is  that  the  precision  of  its 
results  is  limited  by  the  required  maximum  number  of  categories  selected  for  use  in  the 
survey.  (Fink  and  Kosecoff,  1985)  However,  this  is  not  a  limiting  factor  in  the  context  of 
USPACOM's  engagement  activity  selection  problem. 

3.  LEAST  SQUARES  METHOD 

The  least  squares  method  is  the  third  method  presented  by  Anderson  (1987).  The 
least  squares  method  is  a  useful  and  relatively  simple  means  of  obtaining  scaled  interval 
values  from  ordinal  or  comparative  judgments.  The  inputs  of  this  procedure  are  obtained 
by  asking  evaluators  to  do  some  ordinal  ranking  of  various  instances  of  a  selected 
property. 

This  method  is  begun  by  recording  the  responses  of  the  evaluators  comparing 
several  items,  with  respect  to  a  particular  like  characteristic.  For  example,  a  group  of 
senior  supporting  staff  members  might  be  asked  to  rate  three  different  theater  engagement 
activities  for  one  particular  country  in  terms  of  the  overall  impact  on  the  command's 
theater  engagement  plan.  In  this  instance,  one  staff  member  recommends  Activity  B  over 
Activity  C  and  Activity  B  over  Activity  A.  The  responses  of  this  particular  staff  member 
are  contained  in  the  frequency  matrix,/;  shown  in  Table  3.4. 

Since  Activity  B  is  the  preferred  engagement  activity  by  the  senior  staff  member, 
the  matrix  entries  are  made  in  the  corresponding  rows  of  the  column  B  that  were  rated 
inferior  to  Activity  B.  In  this  instance,  entries  are  made  in  rows  A  and  C    Since  Activity 
C  was  the  next  preferred  activity,  an  entry  is  made  in  row  A  and  column  C. 
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fij 

Activity  A 

Activity  B 

Activity  C 

Activity  A 

— 

1 

1 

Activity  B 

— 

— 

— 

Activity  C 

— 

1 

— 

Table  3.4  -  Least  Squares  Method  Scoring  Matrix. 
The  responses  of  all  staff  members  are  recorded  in  this  same  manner,  and 
collected  and  recorded  in  another  frequency  matrix  as  shown  in  Table  3.5.  The  sum  of 
the  cross-diagonal  elements  of  this  matrix  will  be  equal  to  the  total  number  of  the  staff 
members.  For  this  example,  let's  assume  that  there  are  100  staff  members.  After 
reviewing  each  activity  and  evaluating  it  on  its  overall  impact  on  the  command's  theater 
engagement  plan,  54  staff  members  ranked  Activity  A  superior  to  Activity  C  and  46  staff 
members  ranked  Activity  C  superior  to  Activity  A.  Additionally,  72  staff  members 
ranked  Activity  A  superior  to  Activity  B  and  28  staff  members  ranked  Activity  B 
superior  to  Activity  A.  To  complete  the  matrix,  35  staff  members  ranked  Activity  B 
superior  to  Activity  C  and  65  staff  members  ranked  Activity  C  superior  to  Activity  B. 


k 

Activity  A 

Activity  B 

Activity  C 

Activity  A 

— 

28 

46 

Activity  B 

72 

~ 

65 

Activity  C 

54 

35 

— 

Table  3.5  -  Least  Squares  Method  Observed  Frequency  Matrix. 
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The  next  step  is  to  convert  the  frequency  matrix  to  a  probability  matrix.  The 
probability  matrix,/?,;,  can  be  calculated  by  using  Equation  3-4: 

f 
P.  =      — 

"       fr+f- 

J  11  J    It 


(3-4) 


For  the  above  example,  the  probability  matrix  was  obtained  and  is  shown  in  Table  3.6. 
Probabilities  greater  than  0.98  and  less  than  0.02  should  be  omitted  in  order  to  avoid 
numerical  bias. 


Pij 

Activity  A 

Activity  B 

Activity  C 

Activity  A 

0.50 

0.28 

0.46 

Activity  B 

0.72 

0.50 

0.65 

Activity  C 

0.54 

0.35 

0.50 

Table  3.6  -  Least  Squares  Method  Probability  Matrix. 

The  important  thing  to  note  in  Table  3.6  is  that  the  diagonal  entries  of  the 
probability  matrix  are  set  equal  to  0.5.  The  probability  matrix  is  then  converted  to  the 
standard  normal  matrix,  Zy,  by  subtracting  the  mean  value  of  0.5  from  each  value  of  py 
and  dividing  the  differences  by  the  standard  deviation  of  py.  The  Zy  values  are  the 
standard  normal  variables  corresponding  to  the  py  values  of  the  probability  matrix.  In 
regards  to  our  example,  the  Zy  matrix  is  shown  in  Table  3.7.  The  least  squares  estimate 
of  scale  values  s,  was  obtained  by  taking  the  mean  of  each  column  in  the  matrix  Z. 

As  in  the  previous  two  methods,  the  least  squares  estimate  of  scale  values  is  also 
linearly  transformable  to  other  scales.  Survey  and  questionnaire  forms  utilizing  this 
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method  along  with  an  ordinal  rating  scale  are  simpler  and  require  less  time  and  effort  on 
the  part  of  the  evaluators  than  other  methods.  Since  this  method  has  the  advantage  of 
requiring  a  relatively  low  level  or  ordinal  assessments,  the  evaluators  can  simply  list  the 


Zij 

Activity  A 

Activity  B 

Activity  C 

Activity  A 

0.000 

-0.583 

-0.100 

Activity  B 

0.583 

0.000 

-0.385 

Activity  C 

0.100 

0.385 

0.000 

ZZij 

0.683 

-0.198 

-0.485 

Sj=l/n*Z  Zy 

0.227 

-0.066 

-0.161 

Table  3.7  -  Least  Squares  Method  Standard  Normal  Matrix, 
instances  in  the  order  of  importance  regarding  the  compared  factor.  As  shown  in  the 
example,  the  main  disadvantage  of  this  method  is  that  it  requires  a  large  number  of 
evaluators  to  produce  a  reasonably  accurate  probability  matrix.  Although  this  example 
used  100  evaluators,  the  number  of  actual  experts  resident  on  most  military  staffs  is 
significantly  smaller  and  would  preclude  the  use  of  this  method.  Another  issue  resides  in 
the  cases  where  the  evaluators  do  not  rank  all  instances  for  various  reasons,  which  makes 
the  least  squares  procedure  difficult  to  use  in  scale  development. 

4.  CONSTANT  SUM  METHOD 

The  final  method  presented  by  Anderson  is  the  constant  sum  method,  a  method 
developed  by  Comrey  (1950)  that  quantifies  subjective  ratings  using  pairwise 
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comparisons.  In  this  method,  each  activity  is  compared  with  each  other  by  splitting  100 
points.  There  will  be  n(n-l)/2  pairs  that  must  be  considered,  and  100  points  will  be 
divided  between  each  activity  in  accordance  with  an  absolute  ratio  of  the  greater  to  the 
lesser.  For  example,  if  a  staff  member  gives  80  points  to  Activity  A  and  20  points  to 
Activity  B,  this  indicates  that  Activity  A  is  four  times  more  important  than  Activity  B.  In 
the  same  manner  a  split  of  60-40  would  indicate  a  ratio  of  three  to  two,  and  50-50  that 
two  instances  have  the  same  magnitude 

An  example  is  used  to  illustrate  this  procedure.  Suppose  two  staff  members  are 
asked  to  evaluate  three  plans  on  the  basis  of  their  content.  Table  3.8  represents  their 
respective  comparison  matrices  where /ty  is  the  number  of  points  is  given  to  plan  /  when 
compared  with  plan/  Both  staff  members  preferred  plan  A  to  B,  plan  A  to  C,  and  plan  C 
to  B,  but  the  intensities  of  their  endorsements  are  different 


Staff  Member  1 

Activity  B 

Staff  Member  2 

Pij 

ABC 

Pij 

ABC 

Activity  A 

50       20       70 

Activity  A 

50        30       40 

Activity  B 

80        50       60 

Activity  B 

70        50        80 

Activity  C 

70       40        50 

Activity  C 

60        20        50 

Table  3.8  -  Constant  Sum  Method  Comparison  Matrix. 
The  next  step  is  to  construct  a  matrix  V  by  averaging  the  pt]  values  across  evaluators  as 
shown  in  Table  3.9. 
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Vij 

Activity  A 

Activity  B 

Activity  C 

Activity  A 

50 

25 

55 

Activity  B 

75 

50 

70 

Activity  C 

65 

30 

50 

Table  3.9  -  Constant  Sum  Method  Average  Comparison  Matrix. 
Another  matrix  W  is  formed  from  matrix  Vy.  The  Wy  values  are  computed  using  the 
Equation  3-5  and  the  W matrix  for  this  example  is  shown  in  Table  3.10: 


wv  = 


vu 


(3-5) 


Wij 

Activity  A 

Activity  B 

Activity  C 

Activity  A 

1.000 

0.333 

0.846 

Activity  B 

3.000 

1.000 

2.333 

Activity  C 

1.181 

0.428 

1.000 

Table  3.10  -  Constant  Sum  Method  ^Matrix. 
As  shown  in  Equation  3-6,  the  scale  values  can  be  computed  by  taking  the  n    root 
of  each  column  product,  where  n  is  the  number  of  instances  compared  in  this  example. 


The  calculation  and  final  results  of  this  example  are  shown  in  Table  3.1 1. 


(3-6) 
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Si 

=  [(1.000) 

*  (3.000) 

*  (1.181)  ]1/J  = 

=  1.524 

S2: 

=  [  (0.333)  * 

!  (1.000) " 

=  (0.428)  ]1/J  = 

=  0.522 

S3: 

=  [  (0.846)  * 

:  (2.333) " 

:  (1.000)  ]l/i  = 

=  1.254 

Table  3.1 1  -  Constant  Sum  Method  Scale  Value  Calculation. 
The  constant  sum  method  provides  quantitative  values  that  allow  linear 
transformations  and  all  arithmetic  operations.  The  scale,  therefore,  presses  ratio 
properties  rather  than  interval  scale  properties.  Its  advantage  over  the  previous  methods 
lies  in  the  fact  that  all  of  the  quantitative  values  are  on  a  similar  ratio  scale  vice  an 
interval  scale.  When  using  this  method  with  a  large  number  of  instances,  consistency 
becomes  a  problem  due  the  potential  for  self-contradiction  by  the  evaluators. 

Although  each  of  these  methods  present  several  advantages  in  quantifying  subjective 
judgments,  none  of  them,  in  and  of  itself  is  adequate  for  addressing  the  stated  problem. 
The  next  two  sections  will  examine  two  other,  somewhat  more  robust,  methods  of 
performing  this  task. 
B.  MULTI-ATTRIBUTE  UTILITY  THEORY 

Decomposition  of  the  object  to  be  evaluated  into  relevant  dimensions  through  the 
use  of  a  hierarchical,  descriptive  model  is  the  principle  concept  of  Multi-Attribute  Utility 
Theory  (MAUT).  The  lowest  level  hierarchical  dimensions  are  considered  operational. 
This  indicates  that  they  are  measurable  attributes  of  the  evaluation  object.  Each  element 
of  the  hierarchy  is  assigned  a  weight,  which  represents  its  relative  importance  in  the 
overall  hierarchy.  Additionally,  attributes  at  the  lowest  level  are  assigned  utility 
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functions,  which  determine  the  utility  for  each  plausible  value  of  the  attribute. 

The  rigorous  application  of  objective  measurement  to  decision-making  is  the 
approach  attempted  by  utility  theory.  This  decision  analysis  methodology  presupposes 
that  better  decisions  will  result  from  the  rational  and  unbiased  comparison  of  measurable 
attributes  of  performance  alternatives.    An  alternative's  value  is  assumed  to  consist  of 
measures  over  the  criteria  that  contribute  to  its  overall  worth.  All  of  these  values  are 
converted  to  some  common  scale  of  utilities.  (Olson,  1996) 

Utility  theory  assumes  that  the  each  alternative's  performance  on  each  criterion  is 
known  by  decision-makers.  A  single-measure  utility  function  (SUF)  is  used  to  measure 
potential  performance  levels  on  each  criterion.  These  measures  are  generally 
characterized,  as  more  of  a  good  thing  is  better  than  less.  Although  their  measurement 
values  are  considered  continuous,  continuous  scales  of  value  are  not  required  of 
alternatives.  After  identifying  the  single-measure  utility  functions,  the  value  of  each 
alternative  is  measured  by  an  overall  utility  function.  While  this  overall  utility  function 
weights  each  of  the  criteria  of  value,  this  weighting  does  not  necessarily  have  to  be  linear. 
(Olson,  1996) 

The  basic  hypothesis  of  MAUT  states  that  there  exists  a  real  valued  function  U 
defined  on  the  set  of  feasible  alternatives  for  any  decision  problem.  Whether  done 
consciously  or  not,  it  is  this  utility  function  that  decision-makers  wish  to  maximize.  The 
role  of  the  analyst  is  to  determine  the  function  that  aggregates  the  criteria  gi,  g2,..,  gk- 
Bunn  (1984)  believes  that  utility  consists  of  the  following  theoretical  assumptions. 
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•  Structure  -  The  choices  available  to  the  decision-maker  can  be  sufficiently 
described  by  the  payoff  values  and  associated  probabilities  of  those  choices. 
This  implies  that  the  value  of  a  choice  consists  of  the  choice's  measures  on 
factors  of  value  to  the  decision-maker. 

•  Ordering  -  The  decision-maker  can  express  preference  or  indifference 
between  any  pair  of  tradeoffs. 

•  Reduction  of  Compound  Prospects  -  Decision-makers  are  indifferent  between 
compound  prospects  and  their  equivalent  simple  prospects.  This  implies  that 
decision-makers  give  no  value  to  playing  the  game,  or  gambling,  or  working 
in  a  business  that  is  exciting  because  it  is  risky. 

•  Continuity  -  Every  payoff  can  be  considered  to  be  a  certainty  equivalent  for 
the  prospect  of  some  probability/?  of  the  worst  outcome  and  the  inverse 
probability  (1-p).  This  assumption  implies  that  severe  outcomes  would  be 
risked  at  some  small  probability. 

•  Substitutability  -  Any  prospect  can  be  substituted  for  by  its  certainty 
equivalent. 

•  Transitivity  of  Prospects  -  Decision-makers  can  express  preference  or 
indifference  between  all  pairs  of  prospects.  This  extends  the  assumption  of 
ordering  of  payoffs  to  prospects. 

•  Monotonicity  -  For  two  options  with  the  same  payoffs,  decision-makers 
should  prefer  the  option  with  the  higher  probability  of  the  better  payoff. 
Cases  where  this  is  not  demonstrated  imply  that  there  is  some  other  factor  of 
value  that  has  not  been  considered. 

Additionally,  Bunn  indicates  that  there  are  some  caveats  to  utility  theory.  As 

opposed  to  descriptive  measures,  which  describe  how  decision-makers  behave,  utility 

theory  is  normative,  describing  how  decision-makers  should  behave.  One  must  realize 

that  the  behavior  of  many  successful  decision-makers  is  very  much  in  violation  of  the 

above  assumptions.  Utility  values  are  not  necessarily  additive  in  that  U(A  +  B)  may  not 

be  equal  to  U(A)  +  U(B).  This  indicates  the  reasonable  possibility  that  utility  functions 

can  curve.  It  is  important  to  note  that  numerical  scale  of  utility  theory  is  intended  to 

order  preferences,  not  to  measure  their  strengths.    Lastly,  utilities  are  personal  attributes 

representative  of  a  particular  decision-maker  and  do  not  carry  over  to  others.  (Olson, 

1996) 
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Marshall  and  Oliver  (1995)  explain  that  it  is  seldom  possible  for  decision-makers 
to  quantify  every  attribute  in  a  decision  problem.  In  fact,  they  believe  that  as  the  level  of 
the  decision  increases,  the  more  difficult  it  becomes  for  one  to  apply  some  form  of  direct 
measurement.  While  it  is  relatively  simple  to  preferentially  order  a  single  attribute,  it  is 
much  more  difficult  for  decision-makers  to  compare  results  with  multiple  attributes.  A 
MAUT  model  where  each  attribute  is  assigned  a  separate,  nonlinear  utility  function 
allows  each  attribute  to  be  considered  independently  of  the  other  attributes  and  is  defined 
in  the  following  paragraphs. 

The  set  of  all  possible  attribute  results  (/'  =  1,  2,  ...,ri)  is  known  as  Rj.  Assuming 
that  a  preference  ordering  is  defined  on  each  of  these  sets,  the  scalar  utility  from  this 
vector  of  attributes  is  shown  in  Equation  3-7 

U(r)  =  fdkJuJ{rJ)  (3-7) 

7=1 

where  Uj  is  a  utility  function  defined  on  Rj,  kj  is  the  relative  weight  of  attributey,  r,  is  an 
element  of  Rj,  and  r  =  (r/,  r2, ...  rn>).  In  this  particular  description,  along  with  the  first  two 
assumptions  of  basic  utility  theory,  it  is  necessary  to  prescribe  some  additional 
assumptions  to  ensure  that  Equation  3-7  holds. 

The  first  assumption  of  basic  utility  theory  states  that  every  pair  of  elements  in  set 
R  can  be  compared,  and  there  exists  a  well-defined  preference  ordering  of  the  elements. 
So  if  rj  and  r2  are  any  two  elements  of  set  R,  one  should  use  the  following  symbology  to 
define  their  relationships.  Use  rj  >  r2y  if  result  rj  is  preferred  to  r2,  rj  ~  r2,  if  result  r;  is 
equally  preferred  to  r2,  and  r{  <  r2,  if  result  r2  is  preferred  to  r{.  The  second  assumption 
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involves  the  concept  of  transitive  preference  ordering,  which  implies  that  if  one  prefers  rj 
to  r2,  and  r2  to  r3,  then  one  must  prefer  rj  to  r3.  It  is  assumed  that  at  least  one  of  these 
hold  if  rj1}  and  rj2)  are  any  two  results  in  Rj.  The  "Best  Result"  and  "Worst  Result"  for 
any  attribute^  is  denoted  as  r^  and  ry ,  respectively  so  that  r  =  (ri,r2,...,r„ )  and 
r  =  (r, ,  r2 , . . . ,  rn ) .  The  inclusion  of  these  assumptions  is  to  ensure  consistency  when 
comparing  results.  (Marshall  and  Oliver,  1995) 

Additionally,  the  following  four  concepts  are  used  in  determining  the  restrictions 
on  the  preference  ordering  and  must  be  applied  to  ensure  that  the  utility  function 
decompositions  hold: 

1 .  Preferential  Independence  (PI)  -  Given  rj1*  e  Rj,  and  rj2)  e  Rj,  and  rj  e  Rj, 
every  attributey  is  said  to  be  PI  of  attribute  1,  if  rf1*  >  rj2)  for  every  r}. 

2.  Mutual  Preferential  Independence  (MPI)  -  Attribute./  is  said  to  be  MPI  of 
attribute  1,  if/  is  PI  of  1  and  1  is  PI  of/. 

3.  Utility  Independence  -  As  shown  in  the  decision  sapling  in  Figure  3.2,  choose 
any  attribute  set  N,  and  choose  any  three  values  from  Ri,  such  that  rj '  >  rj  > 
rj2) ,  then  choose  any  vector  s  which  is  an  element  of  R2  x  i?j,...,x  R„. 
Attribute  1  is  said  to  be  utility  independent  of  its  complement,  if  the 
indifference  probability/?  stays  unchanged  for  every  possible  s. 

4.  Mutual  Utility  Independence  (MUI)  -  The  subset  of  attributes  and  its 
complement  are  said  to  be  MUI,  if  a  subset  of  attributes  is  utility  independent 
of  its  complement  and  vice  versa.  (Marshall  and  Oliver,  1995) 


<0>    (!„■) 

O   (r,(,),S) 


o 


(1-P) 


^\     (r,(2),s) 


Figure  3.2  -  Diagram  of  Utility  Independence  after  (Marshal  and  Oliver,  1995). 
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It  is  worth  noting  that  the  first  two  concepts  extend  to  the  attribute  subsets  and  the 
third  concept  can  be  extended  to  any  subset  of  attributes  and  its  complement.  The 
assumption  of  utility  independence  is  much  stronger  than  preferential  independence. 
Also,  attribute  set  TV  must  have  the  MPI  property  if  the  decomposition  of  the  utility 
function  into  its  additive  form  is  to  be  considered  valid.  However,  this  is  not  considered 
to  be  sufficient.  The  more  complex  multiplicative  form  is  shown  in  Equation  3-8. 


flll  +  Kkjitjirj)]-! 


^W  =  — ^ (3-8) 

Using  Equation  3-8,  recognize  that  U(r)  =  0  and  U(r)  =  1 .  This  is  necessary  to  scale 
U(r)  on  (0,1).  Also  K  must  satisfy  Equation  3-9 

l  +  K  =  fl(l  +  Kkj).    .  (3-9) 

>i 

One  of  the  important  results  of  MAUT  is  that  the  decomposition  of  Equation  3-8 
only  holds  if  the  set  of //attributes  has  the  property  of  MUI.  This  result  allows  single- 
attribute  utility  functions  to  be  combined  in  the  same  manner  as  multiplicative  utility 
functions  when  MUI  holds.  However,  certain  conditions  must  be  met  concerning  the 
weights  kj  if  the  additive  form  of  Equation  3-8  is  to  be  considered  valid.  (Marshall  and 
Oliver,  1995) 

In  determining  the  k/s,  if  we  let  r(y')  be  the  vector  with  all  attributes  set  to  their 
worst  result  except  they'th,  which  is  set  to  its  best  (for  an  arbitrary  j),  as  shown  in 
Equation  3-10. 

U(r(j))  =  kj  (3-10) 
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By  comparing  the  certainty  of  getting  r(y)to  the  gamble  between  r  and  r  on  a  decision 
sapling,  the  indifference  probability  is  ks.  After  applying  the  decision-maker's  judgment 
in  determining  the  k/s,  one  must  determine  if  these  weights  add  to  1    If  they  do,  then  the 
solution  to  Equation  3-9  results  in  K=  0,  Equation  3-8  simplifies  to  Equation  3-7  and  the 
principle  of  additive  utility  holds.  (Marshall  and  Oliver,  1995) 

If  the  k/s  do  not  add  to  1,  but  the  principle  of  MUI  holds,  Ucan  written  in  terms 
of  the  individual  utility  functions  (u/s)  using  the  utility  function's  multiplicative  form 
(Equation  3-8).  From  this  it  can  be  shown  that 

n 

1 .  If  ^  kj  >  1 ,  K  is  the  unique  root  of  Equation  3-9  in  (-1,0). 

n 

2.  If  ^k <  1 ,  K  is  the  unique  root  of  Equation  3-9  in  (0,oo). 

In  order  to  show  that  N  has  the  MUI  property,  decision-makers  must  check  2n-2  subsets. 
However,  decision-makers  can  reduce  the  amount  of  checking  by  using  the  following 
algorithm.  Find  an  attribute  that  is  utility  independent  of  its  complement  and  number  it  1 
Then  check  to  see  if  each  of  the  (n-l)  pairs  {I  J}  is  preferentially  independent  of  its 
complement.  This  requires  the  making  a  total  of  n  checks  of  which  (n-l)  are  for  the 
simpler  verification  of  preferential  independence  rather  than  utility  independence. 
(Marshall  and  Oliver,  1995) 

Continuing  with  the  theme  of  selecting  the  best  engagement  activity,  the 
following  is  a  simplified  example  of  the  MAUT  process.  In  this  example  Goodwill 
Relations,  Improved  Access,  and  Develop  Coalition  Partner  represent  the  three 
engagement  strategies.  As  shown  in  Table  3.12,  each  of  these  decision  factors  is 
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assigned  a  level  of  relative  importance  on  a  scale  of  0  to  1.  The  sum  of  all  of  these 
decision  factors  must  sum  to  one. 


FACTOR 

IMPORTANCE 

Goodwill  Relations 

0.3 

Improved  Access 

0.5 

Coalition  Partner 

0.2 

Total 

1 

Table  3.12  -  Relative  Importance  of  MAUT  Decision  Factors. 
Each  of  the  engagement  strategies  will  use  a  number  of  alternative  engagement 
activities  to  achieve  its  objective.  As  before,  each  alternative  engagement  factor  (Table 
3.13)  is  assigned  a  relative  importance  on  a  scale  of  0  to  1. 


FACTOR 

GOODWILL 
RELATIONS 

IMPROVED 
ACCESS 

COALITION 
PARTNER 

Port  Visits 

0.8 

0.7 

0.5 

Education 

0.7 

0.4 

0.6 

Exercises 

0.5 

0.8 

0.5 

Conference/Mil  Talks 

0.6 

0.7 

0.5 

Exchange 

0.7 

0.5 

0.7 

High  Level  Visit 

0.8 

0.8 

0.7 

Foreign  Military  Sales 

0.6 

0.6 

0.6 

Table  3.13  -  Relative  Importance  of  MAUT  Alternative  Factors. 
The  weighted  evaluation  for  each  individual  category  is  the  product  of  each  factor 
weighted  and  the  factor  evaluation  of  each  alternative.    Sum  up  each  product  (of  the 
factor  and  factor  evaluation)  to  obtain  the  total  weighted  evaluation  for  a  particular 
alternative.  The  example  shown  in  Table  3.14  shows  the  evaluation  of  the  Port  Visits 
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alternative  as  it  applies  to  the  strategies  of  Goodwill  Relations,  Improved  Access  and 
Develop  Coalition  Partner. 

After  completing  this  process  for  each  alternative  to  be  considered  under  the  three 
engagement  strategies,  compare  the  total  weighted  evaluations.  The  alternative  that  has 
the  highest  total  weighted  evaluation  is  considered  the  best  choice. 


FACTOR  NAME 

FACTOR  WEIGHT 
(IMPORTANCE) 

PORT  VISITS 

FACTOR 
EVALUATION 

WEIGHTED 
EVALUATION 

Goodwill  Relations 

0.30 

0.8 

0.24 

Improved  Access 

0.50 

0.7 

0.35 

Coalition  Partner 

0.20 

0.5 

0.10 

TOTAL 

1 

No  Total  Here 

0.69 

Table  3.14  -  Evaluation  of  Port  Visits  vs.  Individual  Theater  Engagement  Plan 

Categories. 

Although  the  above  example  was  simplified  for  ease  of  computation,  one  can  see 
that  multi-factor  decision  making  can  be  somewhat  complicated.  The  use  of  this 
methodology  requires  decision-makers  to  consider  all  of  the  various  factors  subjectively 
and  intuitively  in  making  the  ultimate  selection.  After  careful  consideration,  all  of  the 
important  factors  can  be  given  appropriate  weights  and  each  alternative  can  be  evaluated 
in  terms  of  these  factors.    Keeny  and  Raiffa  (1993)  provide  more  detailed  information  on 
MAUT. 
C.  ANALYTICAL  HIERARCHY  PROCESS 

The  majority  of  the  background  information  in  this  section  on  the  Analytic 
Hierarchy  Process  (AHP)  is  drawn  from  the  writings  of  Dr.  Thomas  L.  Saaty.  Saaty 
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developed  the  process  in  the  early  1970's  and  has  continually  work  to  improve  it  since 
that  time.  Saaty  has  written  several  books  on  AHP,  co-authored  others,  and  has  also 
authored  numerous  articles  concerning  the  value  and  validity  of  this  process. 
Additionally,  Saaty  and  Dr.  Ernest  H.  Forman  have  developed  a  mature  software  version 
that  has  proven  useful  in  hundreds  of  applications  in  over  27  different  problem  types 
including  determining  requirements,  resource  allocation,  measuring  performance,  conflict 
resolution  and  choice  of  best  policy  alternative.  (Saaty  and  Vargas,  1989) 

Specific  military  examples  of  the  use  of  AHP  and  Expert  Choice™  include 
vendor  selection  by  the  US  Navy  and  the  Joint  Chiefs  of  Staff,  resource  allocation  and 
research  and  development  project  selection  by  the  US  Army  and  force  restructuring 
problems  by  the  Department  of  Defense. 

Saaty  believes  that  the  proper  arrangement  of  the  decision  factors  in  a  hierarchic 

structure  descending  from  an  overall  goal  to  criteria,  subcriteria  and  alternatives  is 

important.  Saaty  (1980)  states  that  "a  hierarchy  is  an  abstraction  of  the  structure  of  a 

system  to  study  the  functional  interactions  of  its  components  and  their  impacts  on  the 

entire  system."  The  decision-maker  must  include  enough  decision  details  to: 

represent  the  problem  as  thoroughly  as  possible,  but  not  so  thoroughly  as  to  lose 
sensitivity  to  change  in  the  elements;  consider  the  environment  surrounding  the 
problem,  identify  the  issues  or  attributes  that  contribute  to  the  solution;  and 
identify  the  participants  associated  with  the  problem  (Saaty,  Management 
Science,  1990). 

The  arrangement  of  the  hierarchy  in  this  manner  provides  an  overall  view  of 

situation's  complex  relationships  and  helps  the  decision-maker  determine  if  the  issues  in 

each  level  are  in  the  same  category  so  he  can  make  accurate  comparisons.    It  is  important 
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to  note  that  these  hierarchies  need  not  be  complete.  An  element  at  a  particular  level  may 
not  necessarily  be  an  attribute  of  all  the  elements  in  the  next  subordinate  level. 
Hierarchies  are  not  the  same  as  traditional  decision  trees  as  each  separate  level  may 
represent  different  views  of  the  same  problem.  (Saaty,  Management  Science,  1990) 

The  next  step  in  the  analytic  hierarchy  process  involves  the  conversion  of 
subjective  judgments  of  the  decision  into  numerical  values.  As  will  be  shown,  AHP  uses 
a  somewhat  different  technique  than  those  previously  discussed.  Saaty  states  that  there 
are  four  major  steps  used  in  this  process: 

1 .  Break  the  decision  problem  into  hierarchical  levels. 

2.  Collect  input  data  by  pairwise  comparisons  of  decision  elements. 

3.  Using  the  eigenvalue  method  to  estimate  the  relative  weights  of  decision 
elements. 

4.  Aggregate  the  relative  weights  at  each  level. 

The  analytical  hierarchy  process  is  better  suited  to  situations  where  decision- 
makers have  difficulty  in  accurately  determining  the  various  factor  weights  and 
evaluations.    A  hierarchy  of  criteria  and  alternatives  allows  decision-makers  to 
decompose  the  information  contained  in  a  decision  problem.  Using  informed  judgments 
to  derive  the  weights  and  priorities,  both  qualitative  and  quantitative  criteria  can  be 
compared.  (Saaty,  1982)  Again,  the  three  most  important  steps  in  the  process  are  to  state 
the  objective,  define  the  criteria,  and  pick  the  alternatives.  Patrick  Harker,  Decisions 
Science  Department,  The  Wharton  School,  University  of  Pennsylvania  states  that  "the 
overall  philosophy  of  the  AHP  is  to  provide  a  solid,  scientific  method  (the  analytic  part) 
to  aid  in  the  creative,  artistic  formulation  and  analysis  of  a  decision  problem  (Golden,  et 
al,  1989)." 
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The  theoretical  foundation  of  AHP  is  based  on  a  set  of  four  axioms.  The  first 
axiom  states  that  given  any  two  alternatives  i  andj  out  of  the  set  A,  the  decision-maker 
will  be  able  to  complete  a  pairwise  comparison  ay  of  the  alternatives  under  any  criterion  c 
from  the  set  of  criteria  C .  Each  element  ay  of  the  pairwise  comparison  matrix  is  the 
reciprocal  of  a,,-,  or  as  shown  in  Equation  3-11. 

a  .=—     \/i,j  =  \,2,...,n  (3-11) 

av 

The  second  axiom  states  that  when  comparing  alternatives,  the  decision-maker  never 

judges  one  to  be  infinitely  better  than  the  other,  or  ay  *  oo  for  all  i,j  in  set  A.  The  third 

axiom  states  that  the  decision-maker  can  formulate  the  problem  as  a  hierarchy.  Lastly, 

the  fourth  axiom  states  that  the  hierarchy  represents  all  of  the  alternatives  and  criteria  that 

impact  on  the  problem.  The  alternatives  and  criteria  must  represent  the  decision-maker's 

intuition  and  should  be  assigned  compatible  priorities.  (Vargas,  1990) 

To  begin  the  process,  the  decision-maker  must  decide  which  of  the  stated 

objectives  is  most  important  as  compared  to  the  others.  This  process  is  known  as 

pairwise  comparisons  and  begins  by  laying  out  the  overall  hierarchy  of  the  decision.  This 

hierarchy  serves  to  reveal  the  factors  for  consideration  as  well  as  various  alternatives  in 

the  decision.  Following  this,  pairwise  comparisons  are  made  which  result  in  the 

determination  of  factor  weights  and  factor  evaluations.  The  alternative  with  highest  total 

weighted  score  is  selected  as  the  best  alternative.  (Saaty  and  Alexander,  1989) 
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Figure  3.3  -  Decision  Hierarchy  for  Engagement  Activity  Selection  after  (Saaty 

and  Vargas,  1982). 

As  depicted  in  Figure  3.3,  the  hierarchical  breakdown  for  selecting  the  best 
engagement  activity  in  this  example  has  three  levels.  The  top  level  describes  the  overall 
decision.  The  second  level  describes  the  factors  (goals/objectives)  that  are  to  be 
considered.  The  lower  level  of  the  hierarchy  depicts  the  alternative  engagement 
activities.  Another  level  below  (not  shown  here)  the  alternatives  would  contain  the 
attributes  of  these  alternatives  or  could  use  some  sort  of  measurement  scale  to  depict  an 
absolute  measure  of  effectiveness  for  each  alternative. 

Each  of  the  three  main  goals/objectives  is  connected  to  each  and  every  one  of  the 
alternative  engagement  activities.  (Saaty  and  Vargas,  1982)  The  use  of  hierarchies 
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allows  the  decision-maker  to  focus  on  each  element  of  the  decision-making  process 
separately.  According  to  Saaty  (Management  Science,  1 990),  it  is  more  effective  for  the 
decision-makers  to  focus  their  judgment  on  a  pair  of  elements  and  use  a  single  property 
for  comparison  between  the  two  without  regard  for  the  other  properties  or  elements. 
Using  pairwise  comparison,  two  alternatives  will  be  evaluated  on  a  relative  scale.  The 
decision-makers  can  define  the  pairwise  comparisons  in  any  manner  they  choose. 

However,  the  prime  consideration  is  that  the  pairwise  comparisons  accurately 
reflect  the  decision-makers'  judgment  on  how  they  want  to  measure  their  alternatives. 
Additionally,  the  pairwise  comparisons  should  be  listed  in  an  ordinal  fashion.  Due  to  the 
large  number  of  pairwise  comparisons  that  would  have  to  be  made  with  large  matrices, 
Saaty  has  recommended  keeping  the  number  of  elements  in  any  level  at  no  more  than 
nine  {1+1-2)  and  the  number  of  levels  between  three  and  five.  It  is  obvious  that  the 
number  of  necessary  pairwise  comparisons  will  increase  exponentially  as  the  numbers  of 
levels  are  increased.  The  pairwise  comparison  definitions  listed  in  Table  3.15  from 
(Saaty  and  Alexander,  1989)  will  be  used  for  the  purpose  of  demonstrating  this 
technique. 

Pairwise  comparison  in  AHP  is  predicated  on  the  premise  that  the  decision-maker 
needs  to  evaluate  a  problem  consisting  of  m  criteria  and  n  alternatives.  As  in  the  constant 
sum  method,  in  order  to  evaluate  m  criteria,  each  decision-maker  must  make  m(m-l)/2 
pairwise  comparisons  (where  m  is  the  number  of  elements  on  a  level  of  the  hierarchy). 
Constructing  a  comparison  matrix,  the  decision-maker  uses  m  rows  and  m  columns  to 
represent  each  attribute.  (Saaty,  1989) 
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As  stated  in  the  axioms,  AHP  requires  that  the  matrix  be  reciprocal.  Additionally, 
the  main  diagonal  elements  of  the  matrix  will  have  the  value  of  unity  (an  =  1,  V7  = 
1 ,2,...m).    The  pairwise  comparison  procedure  is  performed  at  every  level  of  the 
hierarchy,  with  the  exception  of  the  alternatives  themselves,  which  are  the  lowest  level  of 
the  hierarchy.  In  other  words,  if  there  were  one  or  more  levels  of  subcriteria,  pairwise 
comparisons  in  the  manner  described  above  would  be  performed.  In  the  final  or  lowest 
level  of  the  hierarchy,  decision-makers  would  make  a  pairwise  comparison  of  the 
alternatives  one  attribute  at  a  time.  Decision-makers  would  then  evaluate  each  alternative 
by  attribute.  Ultimately,  the  decision-makers  will  have  m  matrices  (one  for  each 
attribute)  of  size  nxn.  (Saaty,  1989) 

The  pairwise  comparison  matrices  are  said  to  be  consistent  if  there  is  a  vector  a  of 
size  n,  in  the  case  of  alternatives  (a  would  be  of  size  m  in  the  case  of  attributes),  such  that 
Equation  3-11  holds.  Otherwise,  the  matrix  is  not  considered  to  be  consistent.    These 
equations  imply  that 

ry=rikxrki     ViJ,k  (3-11) 

for  consistency.  The  vector  a  is  made  unique  through  normalizing  by  dividing  by  its 
sum.  Therefore, 

2>,=1  (3-12) 

If  we  refer  to  the  matrix  of  pairwise  comparisons  as  R;  R  is  consistent  if,  and  only 
if  Ra  =  na,  where  a  =  (a/,  a2,  ....  a„)  represents  the  vector  of  actual  weights  and  n 
represents  the  number  of  elements.  In  a  decision  problem  where  some  inconsistency  is 
present,  AHP  presupposes  that  the  decision-maker  does  not  know  a  and  cannot  produce 
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Numerical  Values 
(Intensity  of  Importance) 

Definition 

Remarks 

1 

Equally  preferred 

Two  activities  contribute 
equally  to  the  objective 

3 

Moderately  preferred 

Experience  and  judgment 
slightly  favor  one  activity 
over  another 

5 

Strongly  preferred 

Experience  and  judgment 
strongly  favor  one  activity 
over  another 

7 

Very  strongly  preferred 

An  activity  is  strongly 
favored  and  its  dominance 
over  the  other  has  been 
demonstrated  in  practice 

9 

Extremely  preferred 

The  evidence  favoring  one 
activity  over  another  is  of 
the  highest  possible  order  of 
information 

2,4,6,8 

Intermediate  values 
between  2  adjacent 
judgments 

Used  when  a  compromise 
between  judgments  must  be 
reached  (consensus) 

Reciprocals 

If  activity  i  has  one  of  the 
above  numbers  assigned  to 
it  when  compared  with 
activity  j,  theny  has  the 
reciprocal  value  when 
compared  to  / 

Used  to  reflect  the 
dominance  first  alternative 
as  compared  with  the 
second 

Rationals 

Ratios  arising  from  scale 

If  consistency  were  forced 
by  obtaining  n  numerical 
values  to  span  the  matrix 

Table  3.15  -  The  AHP  Point  Scale  for  Pairwise  Comparisons, 
the  pairwise  relative  weights  of  matrix  R  accurately.  In  this  instance,  AHP  solves 


*'«'  =  K~«' 


(3-13) 


where  R '  is  the  matrix  of  observed  pairwise  comparisons,  Xmax  is  the  principal  eigenvalue 
of  R ',  and  a '  is  the  right  eigenvector  of  R '  (Zahedi,  1986).  This  leads  to  an 
approximation  of  a,  whose  entries  correspond  to  the  weights  of  the  alternatives  or 


45 


attributes.  To  determine  the  amount  of  inconsistency  and  determine  if  the  amount  of 
consistency  is  acceptable,  Saaty  developed  the  consistency  index  (CI)  defined  as 

a-f=3  (3-i4) 

(n-l) 

where  n  represents  the  number  of  alternatives  being  compared. 

Saaty  explains  that  this  definition  of  consistency  goes  beyond  the  traditional 
requirement  of  preference  transitivity  (if  A  is  preferred  to  B  and  B  is  preferred  to  C,  then 
A  must  be  preferred  to  C)  to  include  the  actual  transitive  intensity  with  which  the 
preference  is  expressed  through  the  comparable  alternatives.  Saaty  defines  the  concept  of 
cardinal  consistency  in  the  strength  of  the  preference  as  follows;  if  A  is  twice  as 
preferable  to  B  and  B  is  three  times  preferable  to  C,  then  A  must  be  six  times  as 
preferable  to  C.  Saaty  states  that  "inconsistency  is  a  violation  of  proportionality  which 
may  or  may  not  entail  the  violation  of  transitivity  (Saaty,  1 980)."    Saaty  further  states 
that  "our  study  of  inconsistency  demonstrates  that  it  is  not  whether  we  are  inconsistent  on 
particular  comparisons  that  matters,  but  how  strongly  consistency  is  violated  in  the 
numerical  sense  for  the  overall  problem  under  study  (Saaty,  1980)."  If  the  CI  <  0.10,  the 
decision-maker  should  accept  the  estimate  of  a.  Otherwise,  the  decision-makers  should 
revise  some  of  their  judgments  during  the  pairwise  comparisons  in  an  attempt  to  improve 
their  consistency.  (Saaty,  1980) 

Decision-makers  could  be  forced  into  consistency  by  making  just  n  pairwise 
comparisons  in  the  first  row  of  the  pairwise  comparison  matrix.  In  this  instance,  the  first 
pairwise  comparison  would  be  alternative  one  compared  to  itself,  which  is  by  definition 
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unity.  Excluding  this  comparison,  (n-1)  pairwise  comparisons  are  all  that  are  necessary. 
In  this  way,  decision-makers  would  obtain  the  entries  for  the  first  row  of  the  pairwise 
comparison  matrix,  and  define  the  weights  based  on  those  entries.  The  entries  for  the  rest 
of  the  matrix  could  be  obtained  by  using  Equation  3-13.  Again,  the  weights  would  be 
normalized  to  sum  to  one.  With  these  weights  obtained  exclusively  from  the  first  row  of 
the  pairwise  comparison  matrix,  every  element  of  the  matrix  could  be  obtained.  The 
resulting  matrix  R  would  be  perfectly  consistent.  (Golden,  et  al,  1989) 

As  indicated,  AHP  does  not  force  this  consistency  on  decision-makers.  By 
requiring  n(n-l)  comparisons,  AHP  makes  the  vector  a  over-determined  and  allows 
inconsistencies.  Therefore,  the  pairwise  comparison  matrix  could  very  likely  contain 
inconsistencies.  With  the  presence  of  these  inconsistencies,  there  is  no  exact  solution  for 
the  vector  a,  such  that  Equation  3-1 1  holds  for  every  i  andy.  The  question  is  how  to  find 
an  a  that  "best"  fits  these  equations  when  inconsistency  is  present.  The  advantages  of  the 
eigenvalue  method  are  (1)  if  the  pairwise  comparison  matrix  is  consistent,  Xmax  =  n  and 
(2)  it  allows  evaluation  of  consistency  by  the  consistency  index  (CI)  defined  above  in 
Equation  3-14.  (Saaty  and  Alexander,  1989) 

As  an  example,  using  the  hierarchy  in  Figure  3.3,  the  decision-makers  begin  at  the 
top  level  of  the  hierarchy  by  looking  at  their  goal/objective  and  comparing  the  relative 
importance  of  Goodwill  Relations,  Improved  Access  and  Coalition  Partner  engagement 
strategies  in  achieving  this  objective.  Using  the  scale  contained  in  Table  3.16,  the  next 
step  will  involve  the  determination  of  the  relative  importance  of  individual  engagement 
activities  under  each  of  the  engagement  strategies.  For  the  simplicity  of  this  example,  we 
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will  assume  that  all  three  of  the  engagement  strategies  are  equally  important.  Under  the 
strategy  of  Goodwill  Relations,  the  decision-makers  determine  that  Port  Visits  are 
extremely  preferred  to  Exchanges  (score  of  9).  The  decision-makers  also  determine  that 
High  Level  Visits  are  very  strongly  preferred  to  Exchange  activities  (score  of  7)  as  a 
measure  of  effectively  conducting  Goodwill  Relations.  Lastly,  the  decision-makers 
determine  that  Port  Visits  are  moderately  preferred  to  High  Level  Visits  (score  of  3). 


Goodwill  Relations 

Port  Visits 

High  Level  Visits 

Exchanges 

Port  Visits 

3 

9 

High  Level  Visits 

7 

Exchanges 

Table  3.16  -  Initial  Pairwise  Comparison  of  Alternatives. 
Based  on  the  information  in  the  preceding  paragraph,  the  AHP  technique  allows 
the  decision-makers  to  determine  the  evaluation  factors  for  the  three  engagement 
activities  under  the  single  goal  of  Goodwill  Relations.  As  shown  in  Table  3.17,  the 
comparison  of  each  alternative  against  itself  results  in  the  assigned  value  of  1,  which 
represents  equally  preferred. 


Goodwill  Relations 

Port  Visits 

High  Level  Visits 

Exchanges 

Port  Visits 

1 

3 

9 

High  Level  Visits 

1 

7 

Exchanges 

1 

Table  3.17  -  Pairwise  Comparisons  of  Alternatives  Against  Themselves. 
In  general,  for  any  pairwise  comparison  matrix,  the  value  of  1  will  be  placed 
down  the  diagonal  from  the  upper  left  corner  to  the  lower  right  corner.  To  finish  the 
matrix,  AHP  makes  the  observation  that  alternative  A  is  twice  as  preferred  to  alternative 
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B,  the  decision-maker  can  conclude  that  Alternative  B  is  preferred  only  half  as  much  as 
alternative  A.  Therefore,  if  alternative  A  receives  a  score  of  2  relative  to  alternative  B, 
then  alternative  B  should  receive  a  score  of  Vi  when  compared  to  alternative  A.  This 
same  logic  is  used  to  complete  the  lower  side  of  the  matrix.  As  defined  by  the  axioms, 
the  values  of  the  lower  side  of  the  matrix  are  the  reciprocals  of  the  value  located  directly 
diagonal  across  their  location.  The  values  should  be  as  depicted  in  Table  3.18. 


Goodwill  Relations 

Port  Visits 

High  Level  Visits 

Exchanges 

Port  Visits 

1 

3 

9 

High  Level  Visits 

1/3 

1 

7 

Exchanges 

1/9 

1/7 

1 

Table  3.18  -  Pairwise  Comparisons  of  Reciprocal  Alternatives. 
The  next  step  in  the  process,  as  shown  in  Table  3.19,  involves  the  conversion  of 
the  fractions  into  decimal  numbers  and  the  summing  of  the  column  totals. 


Goodwill  Relations 

Port  Visits 

High  Level  Visits 

Exchanges 

Port  Visits 

1 

3 

9 

High  Level  Visits 

0.3333 

1 

7 

Exchanges 

0.1111 

0.1428 

1 

Total 

1.4444 

4.1428 

17 

Table  3.19  -  Conversion  of  Pairwise  Comparison  Fraction  into  Decimals. 
Once  each  column  has  been  totaled,  each  number  in  the  matrix  is  divided  by  their 
respective  column  totals  as  shown  in  the  following  table.  Each  column  total  must  equal  1 
(due  to  computational  rounding,  the  results  shown  in  Table  3.20  will  only  approximate  1, 
but  has  no  significant  effect  on  the  final  values). 
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Goodwill  Relations 

Port  Visits 

High  Level  Visits 

Exchanges 

Port  Visits 

0.6923 

0.7241 

0.5294 

High  Level  Visits 

0.2307 

0.2414 

0.4118 

Exchanges 

0.0769 

0.0344 

0.0588 

Total 

0.9999 

0.9999 

1 

Table  3.20  -  Normalization  of  Pairwise  Comparisons. 
To  determine  the  priorities  of  Goodwill  Relations  for  the  three  alternative 
engagement  activities,  simply  find  the  average  of  the  various  rows  from  the  matrix  of 
numbers  as  shown  in  Table  3.21. 


Goodwill  Relations 

Port  Visits 

High  Level 
Visits 

Exchanges 

Row  Average 

Port  Visits 

0.6923 

0.7241 

0.5294 

0.6486 

High  Level  Visits 

0.2307 

0.2413 

0.4117 

0.2946 

Exchanges 

0.0769 

0.0344 

0.0588 

0.0567 

Table  3.21  -  Row  Averaging  of  Pairwise  Comparisons. 
As  shown  in  Table  3.22,  Port  Visits  provides  the  highest  factor  evaluation  as  a 
measure  of  effectiveness  for  Goodwill  Relations.  All  of  the  other  factor  evaluations  for 
the  other  alternatives  can  be  determined  in  the  same  fashion. 


FACTOR 

Port  Visits 

High  Level 
Visits 

Exchanges 

Goodwill  Relations 

0.6486 

0.2946 

0.0567 

Table  3.22  -  Factor  Evaluation  for  Goodwill  Relations. 
Now  that  the  factor  evaluations  have  been  completed  the  decision-makers  must 
conduct  one  additional  process  to  ensure  that  all  of  their  responses  have  been  consistent. 
The  consistency  ratio  can  be  obtained  by  determining  the  weighted  sum  vector  (WSV)  of 
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the  matrix.  Using  the  product  of  factor  evaluation  for  the  first  alternative  (Aj)  from  Table 
3.23  and  the  values  of  the  first  column  (Bji,  Bn,  Bn)  of  the  original  pairwise  comparison 
matrix  (Table  3.19)  initiates  this  process.  After  completing  the  same  process  for  the 
second  and  third  alternative,  sum  these  values  over  the  rows  as  shown  in  Equation  3-15. 


WSV  = 


'{AlXBuh(A2iBn)+-(AjB]ny 
(AjB2l)+(A2XB22)+-(AjB2n) 

(AjBj+{A2XBn2)+-(AjBj 


WS 

ws2 

WS. 


(3-15) 


WSV = 


(0.6486Xl)  +  (0.2946X3)+  (0.0567X9) 
(0.6486X0.3333)  +  (0.2946X0+  (0.0567X7) 
(0.6486X0.1 1 1 1)+  (0.2946X0.1428)  +  (0.0567Xl) 


2.0427 
0.9077 
0.1708 


The  consistency  vector  (CV)  must  be  determined  next.  As  shown  in  Equation  3- 
16,  this  step  is  completed  by  dividing  the  results  of  the  weighted  sum  vector  by  the 
alternative  factor  evaluations  contained  in  Table  3.22. 


CV  = 


W^WS^     ws_ 


=  [c  c2-cn] 


(3-16) 


CV  = 


2.0427     0.9077     0.1708 


0.6486     0.2946     0.0567 


=  [3.1493    3.0811    3.0123] 


Upon  completion  of  these  two  steps,  decision-makers  must  compute  the 
consistency  index  (CI)  and  lambda  (X)  before  determining  the  final  consistency  ratio. 
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Lambda  is  simply  the  average  value  of  the  consistency  vector  and  is  computed  using 

Equation  3-17. 

(A  +  A,  +  ■  •  •  A  ) 
X  =  v  '     ^ *-l  (3-17) 


,   3.1493  +  3.0811  +  3.0123 

X  = =  3.0809 


The  formula  for  determining  the  CI  has  been  previously  defined  as  Equation  3-14.  In  this 
example,  n  =  3,  for  the  three  different  alternatives  being  compared. 

(3.0809-3) 
(3-1) 

Finally,  the  Consistency  Ratio  (CR)  can  be  computed.  The  CR  is  equal  to  the  CI 
divided  by  the  Random  Index  (RI)  as  shown  in  Equation  3-18.  The  RI  is  a  direct  function 
of  the  number  of  alternatives  being  considered.    The  RI  values  in  Table  3.23  were 
obtained  by  computing  the  eigenvalues  for  each  «-size  matrix  using  a  sample  size  of  500. 
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n  value 

RI 

2 

0.00 

3 

0.58 

4 

0.90 

5 

1.12 

6 

1.24 

7 

1.32 

8 

1.41 

9 

1.45 

10 

1.49 

11 

1.51 

12 

1.48 

13 

1.56 

14 

1.57 

15 

1.59 

Table  3.23  -  AHP  Random  Index  Table. 


CR  = 


a_ 

RI 


(3-18) 


C*  =  ^  =  0.0698 
0.58 


For  this  example,  CR  =  0.0698.  The  consistency  ratio  tells  decision-makers  how 
consistent  they  are  with  their  answers.  A  higher  number  indicates  less  consistency, 
whereas  a  lower  indicates  more  consistency.  As  previously  discussed,  generally,  if  the 
consistency  ratio  is  0.10  or  less,  the  decision-makers'  answers  are  relatively  consistent. 
As  shown  above,  the  decision-makers'  answers  in  this  case  were  fairly  consistent  and  the 
results  indicate  that  in  this  instance  Port  Visits  are  considered  the  best  alternative  for 
achieving  the  objective  of  Goodwill  Relations. 
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D.  COMPARISON  OF  MAUT  &  AHP  METHODOLOGIES 

The  MAUT  methodology  is  a  well-developed  technique  for  dealing  with  decision 
problems  containing  risk  and  encompasses  utility  functions  that  are  nonlinear,  such  as, 
additive  linear  on  each  variable,  multiplicative  and  multi-linear.  Also  utility  functions 
have  been  shown  to  be  more  advantageous  when  dealing  with  repetitious  decision- 
making problems.  However,  due  to  rapid  changes  in  the  utility  function  over  time,  it 
must  be  constantly  reevaluated.  This  requires  a  lot  of  time  and  effort  by  decision-makers 
and  does  not  necessarily  lend  itself  for  use  as  a  group  process. 

As  opposed  to  generating  the  utility  function  itself,  the  AHP  methodology  is 
useful  in  generating  the  functional  values  of  a  utility  function.  AHP  gives  scaled  values 
that  are  on  a  ratio  scale,  whereas  MAUT  uses  an  interval  scale.  AHP  provides  a 
mechanism  for  checking  on  the  consistency  of  the  input  data.  As  such,  it  is  not  necessary 
for  AHP  to  assume  consistency  among  the  preferences.  The  MAUT  methodology 
requires  a  transitive  preference  relation  when  building  utility  functions. 

In  recent  years,  several  articles  about  the  disadvantages  of  AHP  and  superiority  of 
MAUT  over  AHP  have  been  published.  Dyer  (1990)  points  out  that  AHP  suffers  rank 
reversal  (an  alternative  that  is  chosen  as  the  best  over  a  set  X  is  not  chosen  when  some 
alternative,  perhaps  an  unimportant  one,  is  excluded  from  X).  He  concluded  that 
changing  ratio  scales  with  interval  scales  as  in  MAUT  could  solve  this  problem.  Another 
method  for  reducing  the  chances  of  rank  reversal  will  be  addressed  in  the  next  chapter. 
Perez  (1995)  gave  the  comparison  of  the  two  methods  and  stated: 
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One  would  expect  MAUT,  since  it  requires  only  the  construction  of  an  interval 
scale,  to  be  suitable  for  a  wider  range  of  applications  than  AHP.  However,  one 
would  also  expect  that  AHP,  since  it  builds  a  ratio  scale,  would  be  more  suitable 
to  some  situations  in  which  the  subjacent  structure  had  a  strong  distributive 
component,  particularly  those  in  which  the  coefficients  of  the  distribution  were 
not  strongly  affected  by  changes  in  the  set  of  available  alternatives. 

The  other  shortcoming  stated  by  Dyer  (1990)  is  the  scaling  method  of  AHP.  The  replies 
to  these  criticisms  by  Saaty  (1990),  Harker  and  Vargas  (1990),  and  the  corresponding 
counter-replies  show  that  no  consensus  on  scaling  has  been  reached. 

The  focus  of  MAUT  and  AHP  is  to  compare  a  defined  set  of  alternatives  against  a 
value  function  that  reflects  attribute  importance.  Both  methodologies  require  decision- 
makers to  logically  structure  a  complex  problem  and  both  appear  to  perform  better  when 
addressing  problems  with  a  limited  number  of  alternatives.    However,  both  can 
adequately  deal  with  relatively  large  numbers  of  attributes.    As  discussed  in  the  next 
chapter,  both  methodologies  have  been  implemented  in  commercial  applications,  which 
make  it  easier  to  solve  and  understand  AHP  and  MAUT  related  decision  problems. 
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IV.      EXPERT  CHOICE™  SOFTWARE  APPLICATION  AND 

EVALUATION 


Expert  Choice™  is  a  software  application  that  supports  decision-making  by 
applying  the  concept  of  the  analytic  hierarchy  process.  This  software  package  provides 
decision-makers  with  the  ability  to  construct  decision  frameworks  (hierarchies)  from 
routine  and  non-routine  decision  problems.  These  frameworks  logically  organize  the 
problem  from  goal  to  criteria  to  subcriteria  down  to  the  alternatives  of  the  decision. 
Expert  Choice™  was  chosen  for  this  analysis  due  to  its  availability,  developmental 
maturity,  problem  type  suitability,  group  applications  and  overall  low  cost. 

As  previously  discussed,  Expert  Choice™,  hereafter  referred  to  as  EC,  is  a 
windows-based,  multi-criteria  decision  support  software  tool  based  on  the  AHP 
methodology.  The  developers  of  EC,  Dr.  Thomas  L.  Saaty  and  Dr.  Ernest  R.  Forman, 
believe  that  AHP  is  a  powerful  and  comprehensive  methodology  capable  of 
accommodating  the  use  of  empirical  data  and  subjective  judgements  in  aiding  decision- 
makers. The  AHP  methodology  assists  decision-makers  in  the  decision  process  by 
allowing  them  to  organize  and  evaluate  the  importance  of  various  objectives  and  the 
alternative  solutions  of  a  decision    In  addition  to  facilitating  all  of  the  aspects  of  AHP 
discussed  in  the  previous  chapter,  EC  allows  decision-makers  to  do  what-if  or  graphical 
sensitivity  analysis.  These  particular  features  allow  decision-makers  to  quickly  evaluate 
the  effect  of  a  change  in  the  importance  of  an  objective  on  the  alternatives  of  choice. 

The  remainder  of  this  section  will  be  used  to  describe  the  operation  and  evaluate 
utility  of  EC  as  an  alternative  selection  software  application.    It  is  not  the  intent  of  this 
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work  to  recreate  the  user's  manual,  but  sufficient  detail  drawn  from  that  source  will  be 
provided  to  show  how  the  application  is  utilized. 
A.  MODEL  DEVELOPMENT 

Modeling  in  EC  can  be  accomplished  by  three  methods.  In  the  first  method 
decision-makers  can  simply  modify  previously  constructed  models  with  similar 
characteristics.  These  models  can  be  stored  in  the  Model  Library.  Initially,  the  Model 
Library  contains  only  the  sample  models  that  are  part  of  the  software,  but  it  can  be  easily 
expanded  as  more  models  are  completed.  In  the  second  method,  decision-makers  can 
construct  the  model  directly  using  the  Evaluation  and  Choice  feature.  In  the  third  and 
final  method,  use  of  the  Structuring  feature  allows  decision-makers  to  structure  a  model 
from  the  "top  down"  or  the  "bottom  up."  Structuring  will  be  discussed  later  in  a  separate 
section.  The  examples  provided  in  this  chapter  use  the  Evaluation  and  Choice  method. 

The  first  step  in  the  creation  of  a  decision  model  is  the  definition  of  the  Goal 
node.  The  Goal  node  is  at  the  top  level  (level  0)  of  the  model.  There  must  be  at  least  two 
or  more  levels  below  this  node  containing  the  criteria  and  the  alternatives,  but  more 
complex  models  can  contain  several  layers  of  criteria  (subcriteria,  sub-subcriteria,  etc.). 
Figure  4.1  shows  a  portion  of  graphical  user  interface  (GUI)  window  of  the  example 
Evaluation  and  Choice  model  screen. 

Using  the  elements  of  the  USPACOM  decision  problem,  the  Goal  node  for  this 
example  is  Select  best  theater  engagement  activity.  Goodwill,  Access,  and  Partner  are 
USAPCOM's  three  major  areas  of  focus  and  represent  the  Criteria  nodes  for  this  goal. 
The  Alternative  nodes  are  contained  in  the  next  lower  level  and  represent  a  sampling  of 
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the  potential  alternatives  that  could  be  considered  by  USPACOM.  The  main  Evaluation 
and  Choice  screen  displays  information  (not  shown  here)  in  the  upper  right  hand  corner 
concerning  the  nodes.  Using  the  currently  selected  node,  decision-makers  have  instant 
access  to  information  about  node  level,  node  number,  global  and  local  priority  of  the 
node,  type  of  synthesis  conducted  on  this  node.    Additionally,  EC  provides  for  the 
definition  of  each  type  of  node  up  to  a  limit  of  65  characters.  The  descriptions  of  the 
criteria  and  alternatives  are  provided  in  Appendix  A. 

In  Figure  4. 1 ,  one  can  also  see  that  the  priorities  for  each  criterion  are  reflected  as 
to  their  relative  importance  to  the  overall  goal.  The  EC  software  will  only  allow  one  to 
view  three  levels  of  the  model  at  a  time  in  this  particular  view.    However,  by  using  the 
Sideways  View  function,  decision-makers  can  see  the  entire  model,  along  with  the 
priorities  of  each  criteria,  subcriteria,  and  alternative  as  it  applies  to  the  goal.    The  model 
shown  in  Figure  4.2  is  the  same  as  that  shown  in  Figure  4.1,  both  of  which  have  only 
three  levels.  Again,  all  of  the  priorities  for  each  individual  criteria  and  alternative  are 
contained  in  Appendix  A. 

When  building  models,  decision-makers  should  keep  in  mind  the  relationship  of 
the  nodes.  There  are  parent  nodes,  children  nodes,  and  other  descendant  nodes.  A  plex  is 
a  branch  of  the  model  headed  by  a  particular  node,  and  includes  all  of  the  descendants  of 
that  node  but  not  the  node  itself.  Leaves  are  represented  by  the  alternatives  and  always 
placed  at  the  bottom  level  of  hierarchy.  An  exception  to  this  applies  to  Ratings  models, 
which  will  be  discussed  later.  Lastly,  peer  nodes  are  the  siblings  of  other  nodes  who  have 
the  same  parent  node.  This  will  become  more  important  later  when  attempting  to  build 
larger,  more  complex  models. 
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Figure  4. 1  -  Evaluation  and  Choice  Model  Screen. 
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GOODWILL 
(-238) 

High  Lev 
(0.182) 

{GOAL 

ACCESS 
(.625) 

Exercise 
(0.251) 

1 

PARTNER 

Port  Vis 

(-136) 

(0.344) 

Conferen 
(0.088) 

Exchange 
(0.050)" 

Educatio 

(0.047) 

Foreign 

(0.037) 

Figure  4.2  -  Sideways  View  of  Evaluation  and  Choice  Model  Screen. 

Alternatives,  also  known  as  "leaves,"  are  added  to  the  model  in  the  same  fashion 
as  descendent  nodes.  Once  decision-makers  have  determined  that  all  of  the  defined 
alternatives  apply  to  any  or  all  of  the  other  corresponding  parent  nodes,  it  is  a  simple 
matter  to  copy  the  entire  set  for  application  to  the  complete  model.  One  can  either  copy 
the  current  node's  to  its  peers,  copy  the  marked  node's  children  to  the  current  node,  or 
copy  the  current  node's  children  to  form  leaves  below  all  other  nodes  at  the  bottom  level 
of  the  model.  In  instances  of  repetitive  information,  EC  also  allows  for  the  copying  of 
complete  plexes  into  other  portions  of  the  model. 

In  addition  to  these  basic  features,  EC  provides  a  Notes  feature  that  allows 
decision-makers  an  opportunity  to  provide  detailed  descriptions  of  the  nodes.  This 
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feature  can  be  utilized  to  provide  additional  information  on  the  current  node  such  as 

explaining  what  information  was  used  to  develop  judgments  concerning  this  particular 

node. 

B.  MODEL  ASSESSMENTS 

Once  the  hierarchy  of  the  model  has  been  constructed,  EC  provides  several 
features  to  accomplish  analysis  of  the  model.  The  first  analysis  to  be  completed  is  the 
assessment.  The  Assessment  Menu  will  allow  decision-makers  assess  the  model  with 
pairwise  comparisons,  data,  what-if,  and  ratings  analysis  features.  In  making  pairwise 
comparisons,  decision-makers  have  the  option  of  selecting  from  three  types  of 
comparison:  importance,  preference,  or  likelihood.  Additionally,  the  actual  pairwise 
comparisons  can  be  made  in  several  forms:  verbal,  graphical  or  numerical. 

Importance  is  normally  used  when  comparing  one  criterion  to  another.  For 
example,  this  model  (Figure  4.3)  indicates  the  Access  alternative  is  moderately  more 
important  than  the  Goodwill  alternative.  The  Preference  method  is  more  appropriate  for 
comparing  alternatives  such  as  determining  that  Port  Visits  are  more  preferable  than 
Conferences  under  the  criterion  Access.  The  Likelihood  method  is  normally  used  when 
comparing  uncertain  events  such  as  the  probability  of  a  combined  exercise  improving 
access. 

Of  the  actual  pairwise  comparison  methods,  the  Verbal  comparison  feature  is  the 
easiest  to  use  and  understand.  This  feature  allows  decision-makers  to  make  comparisons 
of  alternatives  using  a  scale  very  similar  to  one  discussed  in  Chapter  III.  As  shown  in 
Figure  4.3,  this  feature  compares  nodes  using  English  language  terms  and  may  prove 
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very  useful  when  comparing  alternatives  in  terms  of  their  social,  psychological  or 
political  context. 


ACCESS:  Provide  improved  access  for  US 

is  MODERATELY  more  IMPORTANT  than 

GOODWILL:  Foster  goodwill  relations  among  US/countries  in  a  particular  AOR 


EXTREME- 


VERTSTRONG 


STRONG 


MODERATE- 


E3UAL- 


Figure  4.3  -  Verbal  Comparison  Screen. 
This  type  of  comparison  also  contains  a  magnification  feature,  which  allows  decision- 
makers to  make  fine  distinctions  between  two  criteria  that  are  nearly  equal.  This  may  be 
necessary  in  instances  where  decision-makers  feel  that  indicating  that  a  dominant  element 
is  twice  as  dominate  (i.e.,  between  Equal  and  Moderate)  overstates  the  case. 
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The  Numerical  comparison  feature  may  prove  more  useful  when  discussing 
alternatives  from  the  context  of  economical  or  other  measurable  aspects.    Figure  4.4 
contains  an  example  in  the  numerical  matrix  form.  As  one  can  see,  this  representation  is 
not  necessarily  significant  when  dealing  with  individual  decisions,  but  would  possibly  be 
better  suited  when  dealing  with  group  decision-making.  The  group-decision  applications 
of  this  software  will  be  discussed  in  a  subsequent  section. 


With  respect  to  GOAL 


ACCESS:  Provide  improved  access  for  US 

is  3.0  times  (MODERATELY)  more  IMPORTANT  than 
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Figure  4.4  -  Numerical  Matrix  Screen. 
The  information  contained  in  Figure  4.4  is  the  same  as  that  in  Figure  4.3,  but  now 
decision-makers  can  apply  a  numerical  value  to  the  importance  of  one  criterion  versus 
another. 
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As  shown  in  Figure  4.5,  the  Graphical  method  of  comparison  is  somewhat  more 
intuitive,  but  will  require  some  practice  by  decision-makers  to  perfect.  Using  a  pie  chart 
and  a  bar  chart,  the  graphical  comparison  method  allows  decision-makers  to  enter 
judgments  as  a  visual  expression  of  preference,  importance  or  likelihood.  This  feature 
may  prove  to  be  useful  in  instances  where  "fuzzy"  judgments  need  to  be  made  and  must 
be  used  in  cases  where  there  are  only  two  factors. 


GOODWILL:  Foster  goodwill  relations  among  US/countries  m  a  particnlai  AOR 


ACCESS:  Provide  improved  access  for  US 


GOODWILL 


Figure  4.5  -  Graphical  Comparison  Screen. 
Another  screen  shown  in  this  example  is  the  Preliminary  Question  screen.  This 
screen  only  appears  when  decision- makers  have  chosen  the  Verbal  method  of  comparison 
and  only  for  those  comparison  groups  for  which  no  judgments  have  yet  been  entered  or 
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previous  judgements  have  been  erased.  The  Preliminary  Question  screen  allows 
decision- makers  to  do  a  quick  comparison  of  model  elements  by  assessing  whether  one 
element  is  more  important,  equally  important  or  less  important  than  another.  It  is  not 
necessary  for  decision-makers  to  complete  the  preliminary  questions  for  every  pair  before 
moving  to  the  Assessment  screen  in  one  of  the  comparison  modes.  Also,  EC  gives 
decision-makers  the  option  of  skipping  the  preliminary  questions  altogether. 


With  respect  to  GOAL,  is 


GOODWILL:  Foster  goodwill  relations  among  US/conntries  in  a  particular  AOR 

more  IMPORTANT,  equally  IMPORTANT,  or  less  IMPORTANT  than 

ACCESS:  Provide  improved  access  for  US 


Figure  4.6  -  Preliminary  Questions  Screen. 
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After  completing  the  pairwise  comparison  process,  the  next  step  is  calculating  the 
results  of  the  comparison  mode.  Figure  4.7  shows  that  this  calculation  results  in  the 
derived  priorities  of  the  criteria  with  respect  to  the  parent  node. 


Derived  Priorrties-witrrrespeeFtQ  GOAL 


INCONSISTENT  RATIO  =  0.02 


P 


An  Inconsistency  Ratio  of  .1  or  more  may  warrant  some  investigation. 
ACCESS 


.625 

GMm\lL      238 
PARTNER      .136 


Figure  4.7  -  Priorities/Inconsistency  Screen. 
The  priorities  shown  in  Figure  4.7  indicate  that  Access  has  much  more  influence  than  the 
other  two  criteria  in  obtaining  the  overall  goal  of  selecting  the  best  engagement  activity. 
These  priorities  are  local  priorities  in  that  they  sum  to  one.  These  priorities  were  derived 
from  all  of  the  pairwise  comparisons  for  the  current  group  of  elements. 

Another  source  of  important  information  shown  in  Figure  4.7  is  the  Inconsistency 
Ratio.  As  discussed  in  Chapter  HI,  Saaty  professes  that  as  a  general  guideline,  an 
inconsistency  ratio  greater  than  0. 10  is  considered  unacceptable.  The  results  of  this 
example  indicate  that  it  is  well  within  the  limits  established  by  Saaty.  Decision-makers 
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can  address  any  inconsistencies  by  using  the  Reorder  or  Compare  features,  or  they  could 
abandon  all  of  their  previous  judgments  and  start  from  scratch.  The  Reorder  feature 
changes  the  order  of  elements  in  the  comparison  matrix  according  to  descending  priority 
and  restarts  the  comparison  process  with  the  most  important  element  first.    As  elements 
are  shown  in  decreasing  priority  (left  to  right/top  to  bottom)  in  the  Matrix  mode,  any 
judgement  that  appears  out  of  order  is  probably  the  source  of  inconsistency.  The 
Compare  feature  allows  decision-makers  to  reenter  the  comparison  mode  they  were 
previously  in  and  then  modify  one  or  more  judgments. 

The  next  assessment  method  is  the  Data  method.  Data  entry  can  be  used  to  enter 
actual  data  values  such  as  costs,  probabilities,  periods  of  time  or  any  other  values.  This 
feature  allows  decision-makers  to  derive  priorities  for  the  nodes  in  comparison  from  the 
data  rather  than  from  pairwise  comparisons.    This  method  allows  decision-makers  to 
invert  priorities  in  instances  where  a  higher  data  value  is  less  desirable  than  a  lower 
value.  Additionally,  decision-makers  have  the  option  of  converting  their  data  entries  into 
pairwise  comparisons.  The  examples  provided  in  this  work  do  not  use  this  method. 

The  What-If  method  of  assessment  is  used  to  directly  set  the  priorities  using  bar 
graphs.  The  nominal  priority  values  are  established  by  dividing  each  priority  by  the 
largest  priority  value  among  the  group.  Decision-makers  have  the  option  of  changing  the 
nominal  values,  the  priority  values  or  the  bar  lengths.  As  a  priority  value  is  changed,  the 
other  priority  values  are  changed  proportionally.    Once  decision-makers  have  reach  a 
final  set  of  values,  EC  can  easily  save  these  values  to  the  individual  models.  Again, 
examples  of  this  method  will  not  be  provided  here. 
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The  final  assessment  method  to  be  discussed  is  the  Ratings  method.  The  Ratings 
method  combines  the  hierarchy  and  pairwise  comparison  process.  This  method  provides 
a  format  that  allows  decision-makers  to  shift  the  emphasis  of  analysis  from  one  where 
alternatives  are  compared  against  each  other  for  specific  criteria  to  one  where  standards 
are  established  for  the  criteria  and  alternatives  are  compared  against  these  standards. 
Although  this  method  gives  decision-makers  the  capability  of  analyzing  a  larger  number 
of  alternatives,  it  should  only  be  used  when  decision-makers  possess  a  thorough 
understanding  of  their  criteria  and  will  be  able  to  generate  meaningful  scales  for  rating 
the  criteria  and  subcriteria. 

In  contrast  to  the  relative  measurement  of  the  Pairwise  Comparison  method,  the 
Ratings  method  employs  absolute  measurement.  Absolute  measurement  allows  decision- 
makers to  gauge  elements  against  an  established  scale  or  set  of  standards  instead  of 
against  each  other.  Using  this  method,  decision-makers  can  establish  prioritized 
evaluation  categories  into  which  engagement  activities  fall.  An  example  of  these  ratings 
scales  could  evaluate  the  engagement  activity's  impact  on  the  overall  goal  if  implemented 
such  as  Excellent,  Above  Average,  Average,  Below  Average,  Poor.  Another  scale  could 
evaluate  the  engagement  activity's  priority  with  respect  to  the  overall  goal  such  as 
Extremely  Critical,  Highly  Critical,  Moderately  Critical,  Not  Critical  and/or  Extremely 
Significant,  Very  Significant,  Moderately  Significant,  Somewhat  significant,  Not 
Significant.  A  third  example  could  evaluate  the  engagement  activity's  risk  such  as  High 
Risk,  Significant  Risk,  moderate  Risk,  Low  Risk.  These  ratings  scales  or  any  others 
devised  by  decision-makers  can  serve  to  indicate  the  influence  of  a  particular  alternative 
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engagement  activity  on  the  achievement  of  the  overall  goal. 

Although  the  criteria  still  undergo  pairwise  comparisons,  the  alternatives  are 
compared  against  this  previously  established  scale.  One  potential  advantage  of  using  the 
Ratings  method  is  the  inclusion  of  the  Benefits/Costs  Analysis  feature.  This  feature  of  EC 
gives  decision-makers  a  quick,  simple  and  straightforward  method  for  optimizing 
resource  allocation  decisions. 
C.  SYNTHESIZING  JUDGMENTS 

Now  that  the  various  assessment  methods  have  been  discussed,  the  next  process 
concerns  the  synthesizing  of  the  judgments.  Synthesizing  is  the  process  of  weighting  and 
combining  priorities  throughout  the  model  after  all  of  the  judgments  are  made  to  yield  the 
final  result.    Synthesis  determines  the  global  weights  of  the  model's  nodes  by 
multiplying  each  node's  local  priority  by  its  parent's  weight.  These  global  weights  are 
then  summed  to  yield  the  overall  weights.    As  discussed  in  Chapter  III,  the  alternative 
with  highest  weight  is  the  most  preferred  alternative. 

The  Synthesis  process  allows  decision-makers  to  display  their  results  in  summary 
or  detail  form.  The  summary  form  is  shown  in  Figure  4.8  and  indicates  that  Port  Visits 
alternative  is  the  most  preferred  engagement  activity  for  this  example.  The  detailed  form 
of  this  example  is  contained  in  Appendix  A.  Decision-makers  are  also  given  the  option 
of  displaying  their  results  in  the  Ideal  or  Distributive  mode.  This  process  does  not  apply 
to  Ratings  models  as  the  overall  weight  of  an  alternative  is  obtained  from  the  total  score 
column  of  the  spreadsheet. 
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Figure  4.8  -  Synthesis  Summary  Results  Screen. 

The  Ideal  mode  is  useful  when  comparing  several  alternatives  that  are  very 
similar  to  each  other.  This  gives  decision-makers  the  ability  to  prevent  the  weight  of  the 
best  alternative  from  being  diluted  by  its  competitors.  Instead  of  using  normalization,  the 
Ideal  mode  divides  the  numerical  ranks  of  the  alternatives  for  each  criterion  by  the  largest 
value  among  them.  The  most  preferred  alternative  receives  a  value  of  one.  When  new 
alternatives  are  added,  they  are  only  compared  with  the  most  preferred  alternative  for  that 
particular  criterion. 

The  Distributive  mode  is  useful  in  instances  where  the  choice  of  the  best 
alternative  is  influenced  how  many  other  alternatives  exist  and  what  those  other 
alternatives  actually  are.  The  Distributive  mode  distributes  the  weights  of  the  criteria 
among  the  alternatives,  a  process  that  proportionally  divides  the  full  criteria  weights  in  a 
manner  relative  to  the  percentage  of  the  preference  of  the  alternatives.  Discussion  of  this 
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mode  brings  up  the  topic  of  rank  reversal. 

According  to  the  developers  of  EC,  rank  reversal  of  the  final  alternatives  is  only 
possible  in  the  Distributive  mode  and  usually  occurs  when  other  alternatives  or  added  or 
deleted.  When  using  relative  measurement  to  rank  alternatives,  decision-makers  must 
realize  what  happens  to  one  alternative  also  affects  all  of  the  other  alternatives  contained 
in  the  hierarchy.  Inserting  an  additional  alternative  that  bears  a  close  resemblance  to  one 
or  more  alternatives  already  under  consideration  essentially  spreads  the  overall  priority  of 
that  particular  between  both  of  those  alternatives  instead  of  just  one.  When  an  alternative 
is  introduced  that  is  a  duplicate  or  a  close  copy  of  an  existing  alternative,  with  respect  to 
the  established  criteria,  decision-makers  should  revise  the  entire  set  of  criteria  or  delete 
that  particular  alternative.  Since  the  derive  priorities  of  criteria  and  alternatives  are 
specifically  tailored  for  that  particular  decision  problem,  any  change  (adding  new  criteria 
or  alternatives,  deleting  or  changing  others)  essentially  changes  the  problem  and  makes  it 
a  new  one.  A  new  problem  that  must  be  considered  separately  from  the  decision  problem 
that  previously  existed.  (Golden,  et  al,  1989) 

In  either  case,  the  results  of  the  Distributive  or  Ideal  method  are  often  very  similar 
to  each  other.  The  developers  believe  that  the  choice  of  synthesis  methods  depend  on 
whether  the  goal  is  to  select  the  single  best  alternative  (Ideal/open  system)  or  prioritize  all 
of  the  alternatives  (Distributive/closed  system). 
D.  SENSITIVITY  ANALYSIS 

Sensitivity  analysis  can  be  used  by  decision-makers  to  investigate  the  sensitivity 
of  the  alternatives  to  changes  in  the  priorities  of  the  criteria.  Starting  from  the  Goal  node, 
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sensitivity  analysis  shows  the  sensitivity  of  the  alternatives  with  respect  to  the  criteria 
immediately  below  the  goal.  This  allows  decision-makers  to  observe  how  the  overall 
priorities  of  the  alternatives  change  as  the  priorities  of  the  criteria  are  changed.  This 
analysis  can  be  performed  at  lower  levels  of  the  hierarchy  in  models  that  possess  more 
than  three  levels. 

The  Sensitivity  Analysis  mode  provides  decision-makers  with  five  graphical 
options,  each  of  which  emphasizes  different  aspects  of  the  priorities  of  the  criteria  and 
alternatives.  These  five  options  include  Performance,  Dynamic,  Gradient,  2D  Plot,  and 
Difference  sensitivity  analysis.  There  are  two  important  considerations  to  remember 
when  conducting  sensitivity  analysis.  First,  any  changes  in  graphical  mode  are 
immediately  reflected  in  the  other  modes.  Second,  the  sensitivity  analysis  graphs  will  be 
depicted  differently  depending  on  whether  the  type  of  synthesis  mode  is  Ideal  or 
Distributive. 

The  Performance  sensitivity  graph  depicts  all  of  the  information  about  how  the 
alternatives  behave  with  respect  to  each  criterion  and  provides  the  most  compact 
presentation.  The  Performance  sensitivity  graph  also  shows  a  composite  sensitivity 
presentation  of  each  alternative's  overall  performance.  As  shown  in  Figure  4.9,  a  vertical 
line  represents  each  of  the  three  criteria  in  this  example.  As  read  from  the  right  axis,  the 
point  where  the  alternative  line  intersects  the  particular  vertical  criterion  line  represents 
the  priority  received  by  an  alternative,  with  respect  to  that  criterion.  The  overall  priority 
of  the  alternative  is  represented  by  its  intersection  with  the  last  vertical  line,  which  is 
labeled  Overall. 
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Figure  4.9  -  Initial  Performance  Sensitivity  Analysis  Screen. 
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In  this  example,  the  sensitivity  analysis  is  done  using  the  Ideal  synthesis  mode. 
The  results  depicted  in  Figure  4.9  are  consistent  with  previous  examples  discussed  in  this 
chapter.  The  Access  criterion  still  has  the  greatest  influence  of  the  three  criteria  and  the 
Port  Visits  alternative  still  represents  the  best  alternative.    The  results  depicted  in  Figure 
4.10  show  what  occurs  when  decision-makers  change  the  influence  of  one  of  the 
criterion.  In  this  case  the  influence  of  the  Access  criterion  was  reduced  to  approximately 
0.300  vice  0.625  while  the  influence  of  the  Goodwill  and  Partner  criteria  were  increased 
proportionally.  As  the  results  indicate,  the  overall  influence  of  the  Port  Visits  alternative 
is  reduced  and  the  High  Level  Visits  alternative  becomes  more  significant  than  the 
Exercises  alternative.  The  changes  to  the  other  alternatives  appear  to  be  insignificant  in 
this  case. 

The  Dynamic  sensitivity  graph  is  the  next  method  for  performing  sensitivity 
analysis.  This  graph  shown  in  Figure  4.1 1,  places  its  emphasis  on  the  priorities  of  the 
criteria  in  the  model  and  shows  decision-makers  how  changing  the  priority  of  one 
criterion  affects  the  priorities  of  the  alternatives. 
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Figure  4. 1 1  -  Dynamic  Sensitivity  Analysis  Screen. 

As  shown  in  Figure  4.12,  this  sensitivity  graph  also  allows  decision-makers  to 
look  at  the  influence  of  the  individual  criterion  on  each  alternative  by  displaying  their 
components.  Decision- makers  can  examine  the  influence  of  a  criterion  by  simply 
dragging  the  bar  to  the  right  and  observing  how  the  priorities  of  the  remaining  criteria  are 
decreased  proportionally  with  respect  to  their  original  priorities. 
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Figure  4.12  -  Dynamic  Sensitivity  Analysis  Screen  with  Components. 
The  Gradient  sensitivity  graph  provides  a  linear  representation  of  the  alternatives 
against  a  single  criterion.  This  sensitivity  graph  provides  an  emphasis  on  how  the 
alternatives  relate  for  any  priority  assigned  to  the  criterion  shown  on  the  x-axis.  The 
gradient  sensitivity  analysis  lies  in  the  points  where  the  lines  of  the  alternatives  intersect 
with  each  other.  This  type  of  analysis  gives  decision-makers  the  capability  to  identify  the 
"trade-off"  points  where  the  preferred  alternative  with  respect  to  a  selected  criterion 
changes.  Figure  4.13  provides  an  example  of  this  analysis  using  the  Access  criterion. 
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Figure  4.13  -  Gradient  Sensitivity  Analysis  Screen. 
The  2D  Plot  sensitivity  graph  shows  how  well  the  alternatives  perform  with  respect  to 
any  two  criteria.  The  graph  is  divided  into  quadrants  with  each  of  the  particular  criterion 
being  represented  by  the  x-  and  y-axis.  As  shown  in  Figure  4.14,  the  most  favorable 
alternatives  as  defined  in  the  model  will  be  depicted  in  the  upper  right  quadrant,  while  the 
least  favorable  alternatives  will  be  depicted  in  the  lower  left  quadrant.  The  closer  the 
alternatives  are  to  the  corners  (upper  right  -  best,  lower  left  -  worst),  the  more  significant 
their  impact.  Alternatives  depicted  in  the  upper  left  and  lower  right  indicate  potential 
conflicts  between  the  two  criteria  and  decision-makers  may  need  to  conduct  some  trade- 
offs. Figure  4.14  indicates  that  an  opportunity  for  trade-offs  between  the  Exercises 
alternative  and  the  High  Level  Visits  alternative  may  exist. 
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Figure  4.14  -  2D  Plot  Sensitivity  Analysis  Screen. 
The  last  sensitivity  graph  to  be  discussed  is  the  Differences  sensitivity  graph.  In 
this  sensitivity  analysis  method,  one  alternative  is  fixed  for  comparison  against  any  of  the 
other  alternatives.  This  allows  decision-makers  to  compare  criteria  to  see  which  criterion 
accounts  for  the  prioritization  of  the  alternatives.    The  fixed  alternative  is  depicted  on  the 
left  side  of  the  graph.  Alternatives  for  comparison  against  the  fixed  alternative  can  be 
cycled  through  using  the  tabs  above  the  graph.  Looking  at  Figure  4.15,  the  bar  extends  to 
the  left  if  the  alternative  on  the  left  is  the  best  on  that  criterion  and  to  the  right  if  the 
alternative  on  the  right  is  the  best.    This  particular  example  depicts  the  differences 
between  the  Port  Visits  alternative  and  the  Exercises  alternative.    The  results  contained 
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in  Figure  14.5  show  that  the  Port  Visits  alternative  performs  better  than  the  Exercises 
alternative  under  the  Goodwill  and  Access  criteria,  but  performs  worse  than  the  Exercise 
alternative  under  the  Partner  criterion.  Graphs  of  the  remaining  comparisons  are 
contained  in  Appendix  A. 
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Port  Vis  <>  Exercise 


Figure  14.5  -  Differences  Sensitivity  Analysis  Screen. 
E.  GROUP  APPLICATIONS 

1.  Group  Decision  Making 

Group  decision-making  by  its  very  nature  is  a  somewhat  contentious  process,  but 
it  is  also  one  that  can  offer  some  unique  benefits  and  trade-offs.  The  group  decision- 
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making  process  provides  an  environment  for  the  presentation  and  analysis  of  diverse 
perspectives  from  multiple  sources.  In  addition  to  presenting  the  opportunity  for  the 
discussion  of  new  ideas  that  previously  may  not  have  received  consideration,  group 
decision-making  can  sometimes  lead  to  a  more  widespread  acceptance  of  a  decision 
within  an  organization.  Herein  lies  the  value  of  a  decision  support  tool  such  as  Expert 
Choice™.  When  employed  in  a  group  setting,  this  software  application  provides  a 
valuable  tool  for  building  consensus  among  the  participants.  One  drawback  to  this 
approach  is  that  there  may  disagreements  between  participants  and  consensus  may  be 
difficult  to  reach.  However,  it  should  be  noted  that  group  consensus  may  be  difficult  to 
reach  no  matter  what  tools,  procedures,  or  processes  are  used.  Reaching  consensus 
should  be  a  goal,  not  a  requirement.  (Expert  Choice.  1998) 

Expert  Choice  M  addresses  this  issue  by  including  all  inputs,  from  each  group 
member,  in  the  model  during  each  step  of  the  decision-making  process.  Also  included  is 
the  designation  of  importance,  which  is  determined  by  the  group,  not  the  individual.  In 
the  group  decision-making  environment,  "factors  such  as  personal  charisma,  perceived 
intelligence  and  expertise,  and  size  and  strength  of  an  outside  constituency  can  make 
some  participants...'more  powerful'  than  others  (Golden,  et  al,  1989)."  Instead  of 
relegating  themselves  to  playing  a  lesser  role  in  the  decision-making  process,  EC 
manages  the  variability  of  each  individual's  knowledge  by  allowing  the  group  to 
prioritize  the  relative  importance  of  each  decision-maker  in  the  decision-making  process. 
(Expert  Choice,  1998)  Even  in  military  organizations  where  formal  ranks  are  known  and 
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relative  power  is  easy  to  assess,  "these  weights  can  be  used  to  emphasize  judgmental 
contributions  or  to  increase  voting  power  (Golden,  et  al,  1989)." 

The  EC  software  application  breaks  down  the  group  decision-making  process  in 
three  activities,  Brainstorming,  Structuring,  and  Evaluation  and  Choice.  Decision- 
makers use  the  Brainstorming  activity  to  make  lists  of  issues,  objectives,  and  alternatives. 
Staff  members  can  vote  on  items  contained  in  the  list  to  determine  their  relevance  to  the 
decision  problem  being  considered.  The  results  of  this  activity  provide  the  starting  point 
of  the  Structuring  activity.  The  Structuring  activity  can  be  used  to  group  objectives  and 
sub-objectives  into  a  format  that  allows  the  decision  model  to  be  readily  generated.  The 
results  of  this  activity  provide  the  starting  point  of  the  Evaluation  and  Choice  activity.  In 
the  Evaluation  and  Choice  activity  group  members  use  the  process  of  comparison  to 
indicate  their  perceptions  on  the  relative  importance  of  each  objective,  sub-objective  and 
alternative.  As  previously  discussed,  these  comparisons  are  used  to  derive  the  weights  of 
the  objectives  and  alternatives,  which  are  then  synthesized  to  provide  the  results.  Further 
analysis  of  the  impact  of  each  element  upon  the  overall  decision  can  be  conducted 
through  graphical  sensitivity  features  discussed  in  the  previous  section.  The  decision 
model  can  be  completed  by  using  these  activities  in  sequence  or  can  be  built  directly  in 
the  Evaluation  and  Choice  activity.  The  decision  model  examples  provided  in  this 
chapter  were  constructed  using  the  latter  method.  (Expert  Choice,  1998) 

2.  Questionnaire  and  Brainstorming 

This  application  is  a  separate  feature  offered  by  the  developers  of  Expert 
Choice      is  not  included  as  part  of  the  basic  or  group  software  systems.  However,  it  is  a 
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very  useful  tool  when  decision-makers  are  attempting  to  model  complex  decision 
problems  in  a  group  environment.  As  stated  in  the  previous  section,  the  Brainstorming 
activity  is  used  to  create  a  list  of  issues,  objectives  or  alternatives.  Generally,  the  items 
on  these  lists  are  related  to  each  other  in  some  fashion.  An  important  concept  in  using 
this  technique  is  get  as  many  ideas  on  table  as  possible  before  the  evaluation  and 
prioritization  process  begins.  The  Questionnaire  activity  is  designed  to  allow  group 
members  to  provide  their  input  on  specific  questions.  These  questions  may  be 
independent  of  each  other  and  may  be  designed  to  acquire  information  on  a  varied  and 
unrelated  range  of  issues.  Although  these  two  activities  are  designed  to  work 
independently  of  each  other,  carefully  designed  questions  in  the  Questionnaire  activity 
can  enhance  the  results  of  the  Brainstorming  activity.  (Expert  Choice,  1998) 

According  to  the  developers  of  EC,  the  Brainstorming  activity  is  a  capability  that 
provides  a  highly  effective  method  for  obtaining  expert  opinions  about  a  particular 
problem  and  determining  the  initial  direction  of  attack.  This  format  allows  group 
members  the  opportunity  to  consider  a  wide  variety  of  ideas  that  may  not  have  normally 
been  considered.  As  opposed  to  the  traditional  use  of  brainstorming  to  develop  creative 
ideas,  EC's  concept  of  brainstorming  is  intended  to  get  all  of  the  potential  issues  out  in 
the  open  without  attempting  to  determine  their  level  of  significance.  (Expert  Choice, 
1998) 

The  Brainstorming  activity  allows  the  group  members  to  list  all  of  the  potential 
items  or  issues  individually  or  they  may  be  categorized  according  to  some  particular 
characteristic.  At  this  stage,  decision-makers  can  use  the  voting  feature  of  EC  to  pare 
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down  the  list  of  issues  or  categories  to  a  more  manageable  level.  This  feature  may  also 
serve  to  indicate  the  initial  popularity  of  certain  items  or  issues.  It  must  be  noted  that  the 
voting  conducted  by  group  members  at  this  stage  is  only  intended  to  screen  the  list  of 
potential  issues  and  objectives  before  completing  the  model.  Actual  evaluation  of  the 
model  will  be  completed  in  the  Evaluation  and  Choice  activity  after  the  model  is  fully 
constructed.  (Expert  Choice,  1998) 

This  activity  allows  members  of  the  group  to  vote  in  four  different  ways 
depending  on  how  the  issues  or  objectives  are  defined.  In  the  first  method,  each 
participating  member  of  the  group  rates  each  item  on  a  scale  ofl  -  10(10-  most 
important,  1  -  least  important,  0  -  not  voting).  The  next  method  of  voting  involves  a 
simple  indication  of  yes  or  no.  The  third  method  allocates  a  specific  number  of  total 
votes  for  each  participating  member  of  the  group.  Any  number  of  these  votes  may  be 
cast  for  any  of  the  items  or  issues  under  consideration.  However,  once  group  members 
have  used  all  of  their  allocated  votes,  they  may  not  vote  on  any  remaining  items.  For 
instance,  if  a  group  member  is  allocated  10  votes  and  uses  all  of  them  on  the  first  two 
items,  then  he  or  she  cannot  vote  on  the  rest  of  the  items  being  considered.  The  last 
method  of  voting  consists  of  determining  a  particular  number  of  items  that  each  group 
member  may  be  allowed  to  vote  on.  For  example,  each  participating  member  is  only 
allowed  to  vote  4  items.  Once  all  four  votes  have  been  cast,  this  member  is  not  allowed 
to  vote  for  any  remaining  items.  To  ensure  consistency  in  the  results,  once  a  voting 
method  has  been  selected,  this  method  should  be  used  for  voting  on  all  of  the  items. 
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Once  complete,  the  voting  results  are  averaged  for  presentation.  This  method  can  also  be 
used  to  prioritize  or  reduce  a  larger  list  of  alternatives.  (Expert  Choice,  1998) 

The  Questionnaire  activity  is  similar  to  the  Brainstorming  activity  in  that  group 
members  vote  on  the  questions  under  consideration.  However,  the  Questionnaire  activity 
provides  for  the  consideration  of  a  greater  variety  of  ideas  because  it  does  not  require  that 
members  of  the  group  used  the  same  voting  method  for  each  question.  Although  this 
particular  feature  provides  for  greater  flexibility  when  developing  potential  questions,  it 
will  not  allow  the  group  to  make  direct  comparison  of  votes  on  each  of  the  questions 
being  considered.  (Expert  Choice,  1998) 

The  are  six  methods  of  voting  included  in  the  Questionnaire  activity.  The  first 
two  methods,  Rating  on  a  scale  of  1-10  and  yes/no  are  exactly  like  those  methods 
contained  in  the  Brainstorming  activity.  The  third  and  fourth  methods  consist  of 
selecting  from  a  list  of  multiple  choices.  The  third  method  contains  a  list  of  choices 
indicated  from  A-E,  and  the  fourth  method  lists  multiple  choices  numbered  from  1-10. 
The  fifth  and  sixth  methods  are  similar  to  the  third  and  fourth  methods  in  that  these 
methods  consist  of  choosing  from  a  list  of  multiple  choices.  However,  in  these  two 
methods,  each  choice  (A-E  and  1-10,  respectively)  is  assigned  a  weight  that  has  been 
developed  and  designated  by  group  consensus.  The  final  results  of  these  last  two 
methods  are  the  average  of  the  weights  of  the  selected  choices.  (Expert  Choice,  1998) 

Again,  the  results  of  either  or  both  of  the  Brainstorming  and  Questionnaire 
activities  can  provide  the  starting  point  of  the  Structuring  activity. 
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3.  Structuring 

In  the  group  decision-making  process,  the  Structuring  activity  allows  decision- 
makers to  begin  the  analysis  of  a  decision  problem  in  two  different  ways.  The  first 
method  uses  the  alternatives  or  objectives  generated  through  the  use  of  the  Brainstorming 
and  Questionnaire  activities.  These  alternatives  of  choice  are  exported  directly  into  the 
Structuring  activity  for  further  analysis.  The  second  method  consists  of  entering  key 
concepts  and  ideas  into  the  Structuring  "blackboard"  as  they  are  presented  and  discussed 
by  the  group.  From  here,  these  ideas  and  concepts  can  be  organized  by  elements  and 
assembled  into  a  hierarchical  model  and  exported  to  the  Evaluation  and  Choice  activity 
for  evaluation.  (Expert  Choice,  1998) 

If  the  listing  of  alternatives  method  is  chosen,  the  next  process  involves  the 
identification  of  each  alternative's  pros  and  cons.  These  advantages  and  disadvantages 
will  be  translated  and  converted  into  objectives  and  sub-objectives  or  criteria  and  sub- 
criteria.  The  developers  of  EC  believe  that,  in  order  to  derive  the  objectives  from  the 
pros  and  cons,  decision-makers  must  reconceptionalize  pros  and  cons  in  value-neutral 
terms  (instead  of  positive/negative  terms)  and  specify  them  in  a  general  form.  It  is  the 
decision-maker  who  mentally  converts  the  advantages  and  disadvantages  of  an  issue  into 
objectives.  Decision-makers  must  determine  and  evaluate  what  each  alternative's 
advantages  and  disadvantages  means  in  terms  of  the  organization's  overall  goals  and 
objectives.  Expert  Choice™  can  then  be  used  to  automate  the  process  of  determining 
what  important  objectives  need  to  be  met  in  order  to  resolve  the  issue. 
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The  Structuring  activity  also  allows  decision-makers  in  a  group  environment  to 
start  the  construction  of  their  decision  models  from  the  top  down  or  the  bottom  up. 
Constructing  the  model  from  the  bottom  up  begins  with  the  identification  of  alternatives 
at  the  bottom  and  progressing  upward  toward  the  overall  goal.  This  method  of  model 
construction  is  best  suited  for  those  instances  where  decision-makers  understand  the 
alternatives  better  than  the  objectives.  As  previously  discussed,  determining  the  pros  and 
cons  of  the  alternatives  may  aid  in  the  definition  of  the  objectives.  Once  the  objectives 
are  categorized,  they  can  be  evaluated  by  the  Evaluation  and  Choice  activity.  (Expert 
Choice,  1998) 

In  instances  of  strategic  planning  where  the  objectives  are  better  understood  than 
the  alternatives,  top  down  structuring  can  be  used.  Again,  the  resulting  hierarchy  of 
objectives,  sub-objectives,  and  alternatives  can  be  evaluated  by  the  Evaluation  and 
Choice  activity.  (Expert  Choice,  1998) 
F.  SYSTEM/TRAINING  COSTS 

Single  user  versions  of  the  current  release  of  Expert  Choice™  Professional  is 
relatively  inexpensive  at  $1 195.00.  However,  the  Team  Expert  Choice™  25-user  bundle 
packages,  which  include  Team  Keypad,  (group  hardware  and  software),  Internet/Network 
license,  25  participant  licenses,  2  builders  licenses,  1  year  technical  support  and  a  three- 
day  training  seminar  are  somewhat  more  expensive  at  $29,995.00.  The  radio  frequency 
keypad  hardware  used  for  group  environments  to  facilitate  the  remote  input  of  individual 
participants  is  available  separately  from  numerous  other  sources.  Expert  Choice,  Inc. 
offers  a  basic  set-up  consisting  of  1 0  keypads  and  1  radio  frequency  receiver  for 
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$4730.00.  Extra  keypads  are  available  at  the  price  of  S285.00  each.  Additionally,  the 
Questionnaire  &  Brainstorming  Keypad  software  application  is  available  separately  for 
the  cost  of  $4995.00.  Lastly,  the  three-day  training  seminar  ($1495.00  per  person) 
provides  instruction  on  the  use  of  the  group  software  application,  including  all  of  the 
topics  covered  in  this  chapter  and  use  of  the  remote  keypad  technology.  In  an  effort  to 
reduce  overall  training  costs,  this  seminar  could  serve  as  a  "train  the  trainer"  function 
where  only  a  few  individuals  would  need  to  actually  the  attend.  After  completion  of  the 
seminar,  they  would  then  serve  as  organizational  instructors  for  implementation  of 
system. 
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V.       CONCLUSIONS 

The  two  types  of  decision  support  methodologies  represented  in  this  work 
represent  a  portion  of  the  current  state  of  management  science's  efforts  to  assist  decision- 
makers in  solving  multicriteria  decision  problems.  Both  of  these  methodologies  are  well 
founded,  in  theory  and  in  practice,  and  have  existed  for  over  25  years.  These  decision 
support  methodologies  and  their  software  applications,  like  all  other  decision  support 
tools,  are  neither  designed  to  replace  the  decision-maker  nor  diminish  the  responsibility 
for  the  decision  made.  Both  of  them  are  capable  of  representing  a  non-trivial  decision 
process  in  an  effort  to  expand  the  rational  boundaries  of  those  involved  in  the  decision- 
making process.  These  and  other  decision  support  tools  only  serve  to  assist  in  an  analysis 
of  as  much  of  the  available  information  as  the  decision-maker  desires. 

This  decision  support  tool  can  be  used  by  USPACOM  to  more  effectively  and 
efficiently  conduct  the  various  and  multiple  levels  of  the  coordination,  interaction,  and 
decision  making  currently  required  by  the  35  separate  internal  and  external  staff  and 
support  entities  during  the  course  of  the  Engagement  Working  Groups.  Models 
developed  by  the  individual  country  chapters  can  be  imported  and  evaluated  as  a  global 
model  with  respect  to  the  CINC's  overall  goal.  Once  individual  country  models  are 
developed  and  properly  analyzed  with  respect  to  the  appropriate  priorities  and  weights, 
this  tool  can  serve  as  a  method  for  quantifying  the  impact  of  the  most  significant 
engagement  activity.  Additionally  this  decision  support  tool  can  be  useful  in 
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comparatively  assessing  the  value  of  one  activity  type  versus  another  in  the  achievement 
of  a  particular  goal  or  objective. 

As  one  can  observe  from  the  previous  chapter,  the  EC  software  application 
provides  a  simple,  intuitive,  user-friendly  method  for  aiding  decision-makers  in  the 
selection  of  alternatives.  Expert  Choice™  supports  decision-making  by  applying  the 
concept  of  the  analytic  hierarchy  process  and  provides  decision-makers  with  the  ability  to 
construct  decision  frameworks  (hierarchies)  from  routine  and  non-routine  decision 
problems.  These  frameworks  logically  organize  the  problem  from  goal  to  criteria  to 
subcriteria  down  to  the  alternatives  of  the  decision.  Although  the  examples  provided  here 
are  simplistic  in  nature,  it  is  easy  to  visualize  the  power  of  this  tool  if  the  models  were 
populated  with  more  detailed  information  concerning  the  overall  goal,  criteria  and 
alternatives. 

Although  this  decision  support  tool,  or  any  other  currently  available  tool,  does  not 
completely  address  USPACOM's  requirement  to  assess  the  effectiveness  of  engagement 
activities  proposed,  planned  and  conducted  within  its  AOR,  Expert  Choice      does 
provide  a  means  for  quantifying  subjective  judgments  and  assisting  in  the  decision 
process.  Alone,  Expert  Choice™  provides  a  powerful  analytic  tool  for  developing  and 
narrowing  down  a  complex  and  lengthy  list  of  potential  alternatives  and  ultimately 
selecting  the  best  alternative  with  respect  to  the  established  criteria.  Combined  with 
Situational  Influence  Assessment  Modeling  (SLAM)  or  other  influence  modeling  tools, 
Expert  Choice™  should  be  capable  of  providing  actual  measures  of  effectiveness  for 
individual  theater  engagement  activities  and  overall  theater  engagement  plans. 
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In  this  era  of  reduced  budgets  and  closer  scrutiny  of  how  these  scarce  resources 
are  utilized,  the  Cost/Benefit  analysis  feature  of  Expert  Choice™  can  be  used  by 
decision-makers  and  planners  to  analyze  and  maximize  current  resource  allocations. 
Additionally,  the  use  of  Expert  Choice™,  along  with  the  development  and 
implementation  of  USPACOM's  Theater  Engagement  Plan  Management  Information 
System  (TEPMIS),  will  provide  a  mechanism  for  comparing  planned-versus-executed 
engagement  activities  and  the  comparative  value  assessment  of  one  activity  versus 
another  in  the  achievement  of  goals  and  objectives. 
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APPENDIX  A:  EXPERT  CHOICE  (AHP)  SYSTEM  OUTPUTS  AND 

GRAPHS 


The  following  diagrams  represent  the  full,  printable  versions  of  the  GUI  screens 
contained  in  Section  B  of  Chapter  IV.  These  graphs  and  diagrams  also  include:  the 
overall  model  hierarchy;  numerical,  matrix,  graphic,  and  questionnaire  views  of  all  of  the 
criteria  and  alternatives  and  their  respective  comparisons;  synthesis  of  the  leaf  nodes  with 
respect  to  the  overall  goal  (ideal  and  distributive);  and  the  graph  outputs  for  the 
performance,  dynamic,  gradient,  2-D  plot,  and  differences  sensitivity  analysis. 
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Select  best  theater  engagement  activity 


GOAL 


■-  GOODWILL  —              \ 
( -238) 

■  High  Lev 
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■  Exercise 
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(•136) 

■  Port  Vis 
(0.344) 
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(0.088) 

■  Exchange 
(0.050) 

■  Educatio 
(0.047) 

/ 

■  Foreign 
(0.037) 

Abbreviation 

Definition 

GOAL 

ACCESS 

Provide  improved  access  for  US 

Conferen 

Information  exchange  conferences  concerning  regional  issues 

Educatio 

Resident  attendance  of  joint/service/military  war/staff  colleges 

Exchange 

Exhange  of  senior/mid-level  military  officers  for  joint  billets 

Exercise 

Conduct  of  joint/combined  military  training  exercises 

Foreign 

Foreign  military  sales  of  excess  military  equipment 

GOODWILL 

Foster  goodwill  relations  among  US/countries  in  a  particular  AOR 

High  Lev 

Visits/Talks  between  high  level  civilians  and/or  senior  military 

PARTNER 

Develop  coalition  partner  in  a  particular  AOR 

Port  Vis 

Port  Visit  in  AOR  conducted  by  ARG,  CBG,  or  single  ship 

OVERALL  HIERARCHY  VffiW 
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Select  best  theater  engagement  activity 


Compare  the  relative  IMPORTANCE  with  respect  to:  GOAL 
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ACCESS 

4.0 

Row  element  is times  more  than  column  element  unless  enclosed  in  0 


Abbreviation 

Definition 

Goal 

Select  best  theater  engagement  activity 

GOODWILL 

Foster  goodwill  relations  among  US/countries  in  a  particular  AOR 

ACCESS 

Provide  improved  access  for  US 

PARTNER 

Develop  coalition  partner  in  a  particular  AOR 

CRITERION  MATRIX  VIEW 
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Select  best  theater  engagement  activity 
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Abbreviation 

Definition 

Goal 

Select  best  theater  engagement  activity 

GOODWILL 

Foster  goodwill  relations  among  US/countries  in  a  particular  AOR 

ACCESS 

Provide  improved  access  for  US 

PARTNER 

Develop  coalition  partner  in  a  particular  AOR 

CRITERION  QUESTIONNAIRE  VIEW 
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Select  best  theater  engagement  activity 


Compare  the  relative  IMPORTANCE  with  respect  to:  GOAL 


Node:  0 


GOODWILL 


ACCESS 


GOODWILL 


PARTNER 


ACCESS 


PARTNER 


Abbreviation 

Definition 

Goal 

Select  best  theater  engagement  activity 

GOODWILL 

Foster  goodwill  relations  among  US/countries  in  a  particular  AOR 

ACCESS 

Provide  improved  access  for  US 

PARTNER 

Develop  coalition  partner  in  a  particular  AOR 

CRITERION  GRAPHICAL  VIEW 
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Select  best  theater  engagement  activity 

Node:  10000 
Compare  the  relative  PREFERENCE  with  respect  to:  GOODWILL  <  GOAL 


1=EQUAL     3=MODERATE 

5= 

=STRONG 

7= 

VE 

RY  STRONG 

9= 

EXTREME 

1 

High  Lev 

9 

8 

7 

6 

5 

® 

3 

2 

1 

2 

3 

4 

5 

6 

7 

8  I  9 

Exercise 

2 

High  Lev 

9 

8 

7 

6 

5 

4 

3 

2 

1 

2 

3 

® 

5 

6 

7 

8 

9 

Port  Vis 

3 

High  Lev 

9 

8 

7 

6 

5 

4 

® 

2 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Conferen 

4 

High  Lev 

9 

8 

7 

6 

5 

® 

3 

2 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Exchange 

5 

High  Lev 

9 

8 

7 

6 

5 

® 

3 

2 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Educatio 

6 

High  Lev 

9 

8 

7 

© 

5 

4 

3 

2 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Foreign 

7 

Exercise 

9 

8 

7 

6 

5 

4 

3 

2 

1 

® 

3 

4 

5 

6 

7 

8 

9 

Port  Vis 

8 

Exercise 

9 

8 

7 

6 

5 

® 

3 

2 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Conferen 

9 

Exercise 

9 

8 

7 

6 

5 

® 

3 

2 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Exchange 

10 

Exercise 

9 

8 

7 

6 

5 

® 

3 

2 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Educatio 

11 

Exercise 

9 

8 

7 

6 

5 

® 

3 

2 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Foreign 

12 

Port  Vis 

9 

8 

7 

6 

5 

35 

3 

2 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Conferen 

13 

Port  Vis 

9 

8 

7 

6 

5 

35 

3 

2 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Exchange 

14 

Port  Vis 

9 

8 

7 

6 

5 

® 

3 

2 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Educatio 

15 

Port  Vis 

9 

8 

7 

® 

5 

4 

3 

2 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Foreign 

16 

Conferen 

9 

8 

7 

6 

5 

4 

® 

2 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Exchange 

17 

Conferen 

9 

8 

7 

6 

5 

4 

3 

® 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Educatio 

18 

Conferen 

9 

8 

7 

6 

5 

4 

® 

2 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Foreign 

19 

Exchange 

9 

8 

7 

6 

5 

4 

3 

® 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Educatio 

20 

Exchange 

9 

8 

7 

6 

5 

4 

® 

2 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Foreign 

21 

Educatio 

9 

8 

7 

6 

5 

4 

® 

2 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Foreign 

Abbreviation 

Definition 

Goal 

Select  best  theater  engagement  activity 

GOODWILL 

Foster  goodwill  relations  among  US/countries  in  a  particular  AOR 

High  Lev 

Visits/Talks  between  high  level  civilians  and/or  senior  military 

Exercise 

Conduct  of  joint/combined  military  training  exercises 

Port  Vis 

Port  Visit  in  AOR  conducted  by  ARG,  CBG,  or  single  ship 

Conferen 

Information  exchange  conferences  concerning  regional  issues 

Exchange 

Exhange  of  senior/mid-level  military  officers  for  joint  billets 

Educatio 

Resident  attendance  of  joint/service/military  war/staff  colleges 

Foreign 

Foreign  military  sales  of  excess  military  equipment 
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Select  best  theater  engagement  activity 

Node:  10000 
Compare  the  relative  PREFERENCE  with  respect  to:  GOODWILL  <  GOAL 


Exercise 

Port  Vis 
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Foreign 

High  Lev 
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(4.0) 
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Row  element  is times  more  than  column  element  unless  enclosed  in  0 


Abbreviation 

Definition 

Goal 

Select  best  theater  engagement  activity 

GOODWILL 

Foster  goodwill  relations  among  US/countries  in  a  particular  AOR 

High  Lev 

Visits/Talks  between  high  level  civilians  and/or  senior  military 

Exercise 

Conduct  of  joint/combined  military  training  exercises 

Port  Vis 

Port  Visit  in  AOR  conducted  by  ARG,  CBG,  or  single  ship 

Conferen 

Information  exchange  conferences  concerning  regional  issues 

Exchange 

Exhange  of  senior/mid-level  military  officers  for  joint  billets 

Educatio 

Resident  attendance  of  joint/service/military  war/staff  colleges 

Foreign 

Foreign  military  sales  of  excess  military  equipment 
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Select  best  theater  engagement  activity 

Node:  10000 
Compare  the  relative  PREFERENCE  with  respect  to:   GOODWILL  <  GOAL 
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Foreign 


Abbreviation 

Definition 

Goal 

Select  best  theater  engagement  activity 

GOODWILL 

Foster  goodwill  relations  among  US/countries  in  a  particular  AOR 

High  Lev 

Visits/Talks  between  high  level  civilians  and/or  senior  military 

Exercise 

Conduct  of  joint/combined  military  training  exercises 

Port  Vis 

Port  Visit  in  AOR  conducted  by  ARG,  CBG,  or  single  ship 

Conferen 

Information  exchange  conferences  concerning  regional  issues 

Exchange 

Exhange  of  senior/mid-level  military  officers  for  joint  billets 

Educatio 

Resident  attendance  of  joint/service/military  war/staff  colleges 

Foreign 

Foreign  military  sales  of  excess  military  equipment 
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Select  best  theater  engagement  activity 

Node:  20000 
Compare  the  relative  PREFERENCE  with  respect  to:  ACCESS  <  GOAL 
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Abbreviation 

Definition 

Goal 

Select  best  theater  engagement  activity 

ACCESS 

Provide  improved  access  for  US 

High  Lev 

Visits/Talks  between  high  level  civilians  and/or  senior  military 

Exercise 

Conduct  of  joint/combined  military  training  exercises 

Port  Vis 

Port  Visit  in  AOR  conducted  by  ARG,  CBG,  or  single  ship 

Conferen 

Information  exchange  conferences  concerning  regional  issues 

Exchange 

Exhange  of  senior/mid-level  military  officers  for  joint  billets 

Educatio 

Resident  attendance  of  joint/service/military  war/staff  colleges 

Foreign 

Foreign  military  sales  of  excess  military  equipment 
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Select  best  theater  engagement  activity 

Node:  20000 
Compare  the  relative  PREFERENCE  with  respect  to:  ACCESS  <  GOAL 
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Row  element  is limes  more  than  column  element  unless  enclosed  in  Q 


Abbreviation                                                                       Definition 

Goal 

Select  best  theater  engagement  activity 

ACCESS 

Provide  improved  access  for  US 

High  Lev 

Visits/Talks  between  high  level  civilians  and/or  senior  military 

Exercise 

Conduct  of  joint/combined  military  training  exercises 

Port  Vis 

Port  Visit  in  AOR  conducted  by  ARG,  CBG,  or  single  ship 

Conferen 

Information  exchange  conferences  concerning  regional  issues 

Exchange 

Exhange  of  senior/mid-level  military  officers  for  joint  billets 

Educatio 

Resident  attendance  of  joint/service/military  war/staff  colleges 

Foreign 

Foreign  military  sales  of  excess  military  equipment 
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Select  best  theater  engagement  activity 

Node:  20000 
Compare  the  relative  PREFERENCE  with  respect  to:  ACCESS  <  GOAL 


Abbreviation 

Definition 

Goal 

Select  best  theater  engagement  activity 

ACCESS 

Provide  improved  access  for  US 

High  Lev 

Visits/Talks  between  high  level  civilians  and/or  senior  military 

Exercise 

Conduct  of  joint/combined  military  training  exercises 

Port  Vis 

Port  Visit  in  AOR  conducted  by  ARG,  CBG,  or  single  ship 

Conferen 

Information  exchange  conferences  concerning  regional  issues 

Exchange 

Exhange  of  senior/mid-level  military  officers  for  joint  billets 

Educatio 

Resident  attendance  of  joint/service/military  war/staff  colleges 

Foreign 

Foreign  military  sales  of  excess  military  equipment 
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Select  best  theater  engagement  activity 

Node:  30000 
Compare  the  relative  PREFERENCE  with  respect  to:  PARTNER  <  GOAL 
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Abbreviation 

Definition 

Goal 

Select  best  theater  engagement  activity 

PARTNER 

Develop  coalition  partner  in  a  particular  AOR 

High  Lev 

Visits/Talks  between  high  level  civilians  and/or  senior  military 

Exercise 

Conduct  of  joint/combined  military  training  exercises 

Port  Vis 

Port  Visit  in  AOR  conducted  by  ARG,  CBG,  or  single  ship 

Conferen 

Information  exchange  conferences  concerning  regional  issues 

Exchange 

Exhange  of  senior/mid-level  military  officers  for  joint  billets 

Educatio 

Resident  attendance  of  joint/service/military  war/staff  colleges 

Foreign 

Foreign  military  sales  of  excess  military  equipment 
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Select  best  theater  engagement  activity 

Node:  30000 
Compare  the  relative  PREFERENCE  with  respect  to:   PARTNER  <  GOAL 


Exercise 

Port  Vis 

Conferen 

Exchange 

Educatio 

Foreign 

High  Lev 

2.0 

3.0 

2.0 

4.0 

4.0 

2.0 

Exercise 

2.0 

3.0 

4.0 

4.0 

2.0 

Port  Vis 

(2.0) 

2.0 

2.0 

2.0 

Conferen 

3.0 

2.0 

2.0 

Exchange 





2.0 

(2.0) 

Educatio 

(2.0) 

Row  element  is limes  more  than  column  element  unless  enclosed  in  0 


Abbreviation                                                                     Definition 

Goal 

Select  best  theater  engagement  activity 

PARTNER 

Develop  coalition  partner  in  a  particular  AOR 

High  Lev 

Visits/Talks  between  high  level  civilians  and/or  senior  military 

Exercise 

Conduct  of  joint/combined  military  training  exercises 

Port  Vis 

Port  Visit  in  AOR  conducted  by  ARG,  CBG,  or  single  ship 

Conferen 

Information  exchange  conferences  concerning  regional  issues 

Exchange 

Exhange  of  senior/mid-level  military  officers  for  joint  billets 

Educatio 

Resident  attendance  of  joint/service/military  war/staff  colleges 

Foreign 

Foreign  military  sales  of  excess  military  equipment 

107 


Select  best  theater  engagement  activity 

Node:  30000 
Compare  the  relative  PREFERENCE  with  respect  to:  PARTNER  <  GOAL 


Abbreviation 

Definition 

Goal 

Select  best  theater  engagement  activity 

PARTNER 

Develop  coalition  partner  in  a  particular  AOR 

High  Lev 

Visits/Talks  between  high  level  civilians  and/or  senior  military 

Exercise 

Conduct  of  joint/combined  military  training  exercises 

Port  Vis 

Port  Visit  in  AOR  conducted  by  ARG,  CBG,  or  single  ship 

Conferen 

Information  exchange  conferences  concerning  regional  issues 

Exchange 

Exhange  of  senior/mid-level  military  officers  for  joint  billets 

Educatio 

Resident  attendance  of  joint/service/military  war/staff  colleges 

Foreign 

Foreign  military  sales  of  excess  military  equipment 
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Select  best  theater  engagement  activity 


Synthesis  of  Leaf  Nodes  with  respect  to  GOAL 

Ideal  Mode 
OVERALL  INCONSISTENCY  INDEX  =  0.07 


LEVEL  1 

LEVEL  2 

LEVEL  3 

LEVEL  4 

LEVEL  5 

ACCESS  =.625 

Port  Vis=625 

Exerciser  451 

HighLev=.213 

i 

Conferen=.119 

I                                  | 

Exchanges  069 

Educatio=.069 

Foreign  =.042 

GOODWILL=238 

Port  Vis=.  238 

High  Lev=.  172 

Exercise=.119 

Conferen=.061 

Exchange=.043 

Educatio=.037 

Foreign  =.021 

PARTNER  =  136 

High  Lev=.  136 

Exerciser  114 

Conferen=.072 

Port  Vis=.055 

Foreign  =.048 

Exchange=.030 

Educatio=026 

Port  Vis  .333 

Exercise  .248 

High  Lev  .189 

Conferen  .091 

Exchange  .051 

Educatio  .048 
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Select  best  theater  engagement  activity 


Foreign       .0401 


Abbreviation                                           Definition 

GOAL 

ACCESS 

Provide  improved  access  for  US 

Conferen 

Information  exchange  conferences  concerning  regional  issues 

Educatio 

Resident  attendance  of  joint/service/military  war/staff  colleges 

Exchange 

Exhange  of  senior/mid-level  military  officers  for  joint  billets 

Exercise 

Conduct  of  joint/combined  military  training  exercises 

Foreign 

Foreign  military  sales  of  excess  military  equipment 

GOODWILL 

Foster  goodwill  relations  among  US/countries  in  a  particular  AOR 

High  Lev 

Visits/Talks  between  high  level  civilians  and/or  senior  military 

PARTNER 

Develop  coalition  partner  in  a  particular  AOR 

Port  Vis 

Port  Visit  in  AOR  conducted  by  ARG,  CBG,  or  single  ship 
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Select  best  theater  engagement  activity 


Synthesis  of  Leaf  Nodes  with  respect  to  GOAL 

Distributive  Mode 
OVERALL  INCONSISTENCY  INDEX  =  0.07 


LEVEL  1 

LEVEL  2  I                  LEVEL  3|                  LEVEL  4  j                  LEVEL  5 

ACCESS  =625 

i 

Port  Vis=  246 

Exerciser  1 77 

High  Lev=. 084 

i 

Conferen=.047 

Exchanges  027 

Education  027 

Foreign  =.016 

GOODWILL=238 

Port  Vis=.082 

High  Lev=.059 

Exerciser  041 

Conferen=.021 

Exchange=.015 

Educatio=.013 

Foreign  =.007 

PARTNER  =136 

High  Lev=. 039 

Exerciser  032 

Conferen=020 

PortVis=.016 

Foreign  =  014 

Exchanges  009 

Educatio=.007 

Port  Vis  .344 

Exercise  .251 

High  Lev  .182 

Conferen  .088 

Exchange  .050 

Educatio  .047 
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Select  best  theater  engagement  activity 


Foreign       .0371 


Abbreviation 

Definition 

GOAL 

ACCESS 

Provide  improved  access  for  US 

Conferen 

Information  exchange  conferences  concerning  regional  issues 

Educatio 

Resident  attendance  of  joint/service/military  war/staff  colleges 

Exchange 

Exhange  of  senior/mid-level  military  officers  for  joint  billets 

Exercise 

Conduct  of  joint/combined  military  training  exercises 

Foreign 

Foreign  military  sales  of  excess  military  equipment 

GOODWILL 

Foster  goodwill  relations  among  US/countries  in  a  particular  AOR 

High  Lev 

Visits/Talks  between  high  level  civilians  and/or  senior  military 

PARTNER 

Develop  coalition  partner  in  a  particular  AOR 

Port  Vis 

Port  Visit  in  AOR  conducted  by  ARG,  CBG,  or  single  ship 

112 


Performance  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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Dynamic  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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Gradient  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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Two-D  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 

ACCESS 


.40 


.10 


u00 


mm 


iLev 


•rTTirrrirmr? 


.10 


.20  .30 

GOODWILL 


.40 


116 


Differences  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 


Port  Vis  <>  Exercise 
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Weighted  differences  between  Port  Vis  and  Exercise 
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Differences  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 


Port  Vis  <>  High  Lev 
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Weighted  differences  between  Port  Vis  and  High  Lev 
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Differences  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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Weighted  differences  between  Port  Vis  and  Exchange 
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Differences  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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Weighted  differences  between  Port  Vis  and  Educatio 
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Differences  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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Differences  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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Weighted  differences  between  Port  Vis  and  Foreign 
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Differences  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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Differences  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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Weighted  differences  between  Exercise  and  Exchange 
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Differences  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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Differences  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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Weighted  differences  between  Exercise  and  Conferen 
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Differences  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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Weighted  differences  between  Exercise  and  Foreign 
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Differences  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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Differences  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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Differences  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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Differences  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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Weiahted  differences  between  Hiah  Lev  and  Foreian 
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Differences  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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Weighted  differences  between  Exchange  and  Educatio 
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Differences  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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Differences  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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Differences  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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Differences  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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Differences  Sensitivity  w.r.t  GOAL  for  nodes  below  GOAL 
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